---
title: "The Same Detection Rates Can Produce Very Different AI Alert Precision"
description: "Leviathan Systems — Technical content Published 2026-10-21; updated 2026-10-21."
canonical: "https://www.leviathansystems.co/articles/ai-alert-precision-positive-class-prevalence"
last-updated: "2026-10-21"
---

# The Same Detection Rates Can Produce Very Different AI Alert Precision

Leviathan Systems — Technical content
Published 2026-10-21; updated 2026-10-21.

True-positive and false-positive rates alone cannot establish the fraction of positive AI alerts that are useful. They describe outcomes within the reference-positive and reference-negative classes; precision describes the composition of the emitted positive alerts. The classes' prevalence connects those quantities. In the fictional RATE-31 example, identical 90% detection and 10% false-positive rates produce precision of 1/12 in one population and 1/2 in another. A buyer needs evidence applicable to its positive definition, incoming population and decision condition before treating a reported precision as support for its requirement.

## Define the item and the positive reference class

Start with the outcome the buyer wants an alert to identify. In RATE-31, a positive reference label means an item requires follow-up under an invented assessment rule. A negative reference label means it does not. Calling a positive alert useful here means that it corresponds to a reference-positive item. Whether the eventual follow-up resolves a problem is a further outcome.

Every item has a reference label and receives one final binary decision: emit a positive alert or make a negative decision. A true positive, TP, is a reference-positive item that receives an alert. A false negative, FN, is a reference-positive item that receives a negative decision. A false positive, FP, is a reference-negative item that receives an alert. A true negative, TN, is a reference-negative item that receives a negative decision.

For matrix notation, use reference classes as rows and predicted classes as columns, both ordered negative 0, positive 1. The resulting layout is [[TN, FP], [FN, TP]]. This follows the scikit-learn 1.7.2 confusion-matrix reference. A report using another axis convention needs its own explicit mapping.

The calculations use ordinary unweighted binary counts. A multiclass average or a weighted score would require a different interpretation.

References: [CM1](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.confusion_matrix.html)

## Read each rate against its denominator

True-positive rate is TPR = TP / (TP + FN). Its denominator includes every reference-positive item, whether alerted or missed. In this binary task it is also recall for the positive class. A 90% TPR means nine out of ten reference positives receive positive decisions.

False-positive rate is FPR = FP / (FP + TN). Its denominator includes every reference-negative item. A 10% FPR means one out of ten reference negatives receives a positive decision.

Precision is TP / (TP + FP), also called positive predictive value. Its denominator contains all positive decisions. It answers the buyer's question: among the emitted alerts, what fraction correspond to reference positives? The scikit-learn 1.7.2 precision reference defines this ratio and distinguishes the positive-class binary result from other averaging modes.

The same TP count appears in detection and precision, but the other terms differ. Detection compares found positives with all positives. Precision compares found positives with everything alerted. Consequently, a detection percentage cannot be substituted for the useful-alert fraction. Likewise, FPR measures false alerts within the negative class; the fraction of emitted alerts that are false is FP / (TP + FP).

References: [PR1](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.precision_score.html), [CM1](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.confusion_matrix.html)

## RATE-31 X: nine useful alerts among 108 alerts

RATE-31 constructs two fictional populations of 1,000 items each. Every reference label and decision is stipulated. At one fixed classification condition T, including the decision threshold, stipulate TPR = 9/10 and FPR = 1/10 in both populations. These are exact arithmetic constructions; no classifier was executed and no operational result is being reported.

Population X contains 10 reference positives and 990 reference negatives. Detecting nine of the 10 positives gives TP = 9 and FN = 1. Flagging one tenth of the 990 negatives gives FP = 99 and TN = 891. The four cells account for all 1,000 items.

The positive-alert pool contains TP + FP = 9 + 99 = 108 items. Only nine correspond to reference positives, so precision is 9/108 = 1/12, approximately 8.33%. Its false-alert fraction is 99/108 = 11/12, approximately 91.67%, even though its false-positive rate is 99/990 = 10%.

Accuracy, the fraction of all decisions matching the reference, is (TP + TN) / 1,000 = (9 + 891) / 1,000 = 90%. That percentage still leaves the buyer with only nine useful positives in the 108 emitted alerts.

## RATE-31 Y: 90 useful alerts among 180 alerts

Population Y contains 100 reference positives and 900 reference negatives. Under the same stipulated conditional rates, TP = 90 and FN = 10, while FP = 90 and TN = 810. These four cells again account for all 1,000 items.

Its positive-alert pool contains 90 + 90 = 180 items. Precision is 90/180 = 1/2, or 50%. Its TPR remains 90/100 = 90%, and its FPR remains 90/900 = 10%. Accuracy is also unchanged: (90 + 810) / 1,000 = 90%.

The buyer's fictional requirement is that at least half of emitted positive alerts correspond to reference positives. X fails that arithmetic requirement; Y meets it exactly. Both can truthfully display the same detection, false-positive-rate and accuracy headlines.

Y emits more alerts than X, but the useful share is higher. X has fewer reference positives available to contribute true alerts and more reference negatives available to contribute false alerts. The equal conditional rates act on those different class totals. That is enough to change the composition of the positive-alert pool.

## Derive the connection between prevalence and precision

Let N be the total item count and pi be the fraction of items in the positive reference class. Positive prevalence is measured before dividing items by their predicted decisions. The positive and negative class counts are pi × N and (1 − pi) × N.

Write t for TPR and f for FPR. From the rate definitions, TP = t × pi × N and FP = f × (1 − pi) × N. Substituting both into TP / (TP + FP) and cancelling N gives precision = t × pi / [t × pi + f × (1 − pi)], provided the denominator is nonzero.

For X, pi = 0.01. Precision is 0.009 / (0.009 + 0.099) = 1/12. For Y, pi = 0.10, giving 0.09 / (0.09 + 0.09) = 1/2. The formula reproduces the complete counts rather than supplying a separate performance claim.

It also sharpens the fictional buying requirement. With t = 0.9 and f = 0.1 held fixed, at least half the alerts are true positives exactly when 0.9 × pi is at least 0.1 × (1 − pi). Rearranging gives pi at least 0.10. The requirement therefore cannot be assessed from those two conditional rates without the class mix.

If no positive decisions exist, the precision denominator is zero. A reporting package's configured zero-division value cannot establish the quality of an alert pool that contains no alerts.

References: [PR1](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.precision_score.html)

## Establish which population the report describes

Ask what was eligible to enter the evaluation and what counts as one item. The report should identify the positive reference rule, reference-label process, observation period, evaluated model or system revision and final decision condition. An item entering the system, an emitted alert and a completed follow-up are different units; the numerator and denominator must belong to the unit named in the requirement.

A test collection deliberately enriched with positives can be useful for examining detection. Its directly calculated precision describes that enriched collection. If the buyer's incoming population has fewer positives, that test precision does not by itself establish the incoming useful-alert fraction. RATE-31 shows the consequence even under equal conditional rates.

Request the underlying TP, FN, FP and TN counts alongside the rates. They expose the evaluated class proportions and the number of positive decisions. Examining emitted alerts can support a precision estimate for those alerts if the reference judgments and selection are appropriate. It leaves the unalerted population unresolved, so it cannot alone establish the complete class mix, TPR or FPR.

RATE-31 holds t and f fixed to isolate prevalence. A real change in population can also change the kinds of positive and negative items and therefore either conditional rate. Evidence from the buyer's representative population at the relevant decision condition is needed to assess applicability. Finite evaluation counts and uncertain reference judgments also need consideration; an observed fraction at the requirement boundary does not guarantee that future work will meet it.

## Use the report to resolve the actual requirement

Suppose a supplier presents Y's complete constructed counts as its illustrative report, while the buyer's proposed population has X's 1% positive prevalence. Y demonstrates 50% precision on Y. The two conditional-rate headlines alone leave the buyer's required 50% precision unestablished. Applying those same stipulated rates to X produces 1/12, showing why the population question changes the purchasing interpretation.

A usable positive-alert quality claim identifies the assessed population, agreed meaning of positive, system revision and decision condition, then reports the useful positives and total emitted positives on that basis. Supporting evidence should explain why that basis represents the work the buyer intends to submit. Where those elements are missing or materially different, narrow the claim to what was assessed or obtain applicable evidence before relying on it.

Keep any separate missed-positive requirement visible. Y's 50% precision coexists with 10 missed positives; X's lower precision coexists with one missed positive in its smaller positive class. Precision alone cannot settle an overall quality requirement. The related AI review cost analysis examines retained and missed important findings alongside the quality of emitted alerts.

This decision concerns final binary classifications against a reference. A class-score value, a language-model likelihood or nearest-neighbor recall measures another quantity. Use the related retrieval guide when the report's recall instead concerns agreement with an exhaustive vector result.

## Scope an independently needed equipment installation

If the selected application also creates a separately identified equipment installation requirement, Leviathan's rack-and-stack service provides a route to discuss agreed equipment mounting, physical IT cabling, labels and installation records. Use the related service and project inquiry with a nonconfidential equipment list, site and released rack/cable package to discuss scope and availability. Application evaluation and the alert-quality decision remain with the appointed software team.

References: [L1](https://www.leviathansystems.co/services/rack-and-stack)

## Primary sources

- [PR1 — scikit-learn 1.7.2 — precision_score](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.precision_score.html). Checked 2026-10-21. Defines positive-class binary precision, distinguishes averaging modes and recall, and explains zero-division handling. RATE-31 uses unweighted counts and independently derived arithmetic.
- [CM1 — scikit-learn 1.7.2 — confusion_matrix](https://scikit-learn.org/1.7/modules/generated/sklearn.metrics.confusion_matrix.html). Checked 2026-10-21. Defines reference-class rows and predicted-class columns, including the binary TN, FP, FN and TP cells. The article declares negative 0 and positive 1 and uses raw counts.
- [L1 — Leviathan Systems — Rack-and-Stack & Colocation Installation](https://www.leviathansystems.co/services/rack-and-stack). Checked 2026-10-21. Agreed physical equipment mounting, IT cabling, labels and installation records, applicable to a separately identified equipment project.

## Related Leviathan resources

- [AI review costs: retained findings, missed problems and alert quality](/blog/cloudflare-ai-review-cost-session-work-quality)
- [Vector retrieval recall against an exhaustive reference](/articles/vector-search-exact-versus-approximate-retrieval)
- [Rack-and-stack installation scope](/services/rack-and-stack)
- [Discuss a separate physical equipment installation](/contact?service=rack-and-stack&from=%2Farticles%2Fai-alert-precision-positive-class-prevalence)
- [The Same Convolution Weights Can Produce Different Boundary Features](/articles/ai-convolution-padding-boundary-feature-values)

## Does selected equipment need physical installation?

Share a nonconfidential equipment list, site and released rack/cable package to discuss physical installation scope and availability.

[Discuss the scope](/contact?service=rack-and-stack&from=/articles/ai-alert-precision-positive-class-prevalence)
