From Sensitivity to Cost per Case Detected: Predictive Values, Prevalence and the Economics of Screening

อ่านฉบับภาษาไทย (Thai version)

Abstract

A district committee is offered an AI tool to screen its 10,000 adults once for a disease that is silent early. In the company's study of 178 people, the tool flagged 22 of the 58 who had the disease and cleared 98 of the 120 who did not. Half of the flagged people had the disease. In the district only 2 in 100 adults have it, so the same tool flags 1,873 people and about 76 of them are sick. A positive result now means disease 4.05% of the time, while the tool's accuracy rises from 67.42% to 80.79%. Testing everyone and confirming every positive costs 5,746,000 THB, or 75,605 THB per case found, and 62.55% of it pays for false alarms. Finding a case is not helping a person. Earlier diagnosis, slow disease and harmless disease all flatter screening, and the committee's own rule for weighing false alarms shows this tool doing worse than no screening.

🔍 Click to enlarge Click anywhere to close

An AI tool, a company's study and 10,000 adults to screen

A district health committee must decide whether to screen its 10,000 adults once for a disease that causes no symptoms in its early stage. A company offers an AI tool that reads a quick scan taken at the village health centre and flags people who may have the disease. Each flagged person would then go to the hospital for a confirmation test that settles whether they really have it.

The company's brochure reports one study of 178 people at a hospital: 58 known to have the disease and 120 known not to. The tool flagged 22 of the 58 and missed the other 36. It cleared 98 of the 120 and flagged the other 22 by mistake. These numbers are illustrative, for teaching, not results of a real study.

The brochure's headline is that the tool classified 67.42% of the people correctly. One committee member adds that in the district, where few people have the disease, it would do even better. The committee wants two simpler answers: if a villager is flagged, how likely is it that they have the disease, and what will the programme cost for each case it finds?

About 2 in 100 of the district's adults have the disease today. That share of a population with the disease at one time is its prevalence (Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models). In the study it was much higher, and that one difference changes almost every answer below.

What the study can tell us: sensitivity and specificity

Read first on this site:

• What sensitivity and specificity measure in a diagnostic accuracy study
• Sensitivity, specificity, predictive values and the other diagnostic indices

The study sorted people by disease status before testing them, so it can say how often the tool gets each group right. Among the 58 people with the disease, the tool flagged 22, or 37.93%: its sensitivity, the share of people with the disease that it flags. Among the 120 without the disease, it cleared 98, or 81.67%: its specificity, the share of people without the disease that it clears.

A flag is a positive result and a clearance a negative one. So the 22 flagged people with the disease are true positives (TP), and the 36 it missed are false negatives (FN). The 22 flagged people without the disease are false positives (FP), or false alarms, and the 98 it cleared are true negatives (TN).

For the committee, these two shares are the part of the study that can travel to the district. Each is worked out inside one group, so it does not depend on how many of the people tested are sick. As a later section shows, it can still depend on how sick they are. A sensitivity of 37.93% is low: the tool misses about 6 in 10 people who have the disease, and the district will count them.

Accuracy: the brochure's headline number

In the study the tool got 22 + 98 = 120 of the 178 people right, which is the brochure's 67.42%. This share of all tested people classified correctly is called accuracy.

To see the district, follow its 10,000 adults through the tool, counting people instead of juggling percentages. For now the tool keeps the sensitivity and specificity the study measured, an assumption we question later.

Why accuracy rose: a weighted average

Accuracy mixes the two groups' scores, each weighted by the size of its group. In the study, about a third of the people had the disease (58 of 178 people, 32.58%), so the weak sensitivity counted for a third: 0.3258 × 37.93% + 0.6742 × 81.67% = 12.36% + 55.06% = 67.42%. In the district, 98 in 100 adults are well, so the specificity nearly decides the accuracy alone.

With Se for sensitivity, Sp for specificity and π (the Greek letter pi) for the prevalence:

$$Acc=Se\,\pi+Sp\,(1-\pi)$$

In the district that is (22/58) × 0.02 + (98/120) × 0.98 = 0.76% + 80.03% = 80.79%. That rise is a warning, not good news: a tool that clears well people better than it flags sick ones looks more accurate the rarer the disease, without finding a single extra case. Accuracy alone should never be the headline.

What a positive result means: predictive values

Read first on this site:

• From a test result to a clinical judgement: predictive values and the chance after a test

The flagged villager asks a different question: now that I am flagged, how likely is it that I have the disease? That question starts from the flagged group, not from the sick group.

In the study, 44 people were flagged and 22 of them had the disease: 50.00%. In the district, 1,873 are flagged and about 76 have it. Before rounding, the flagged are 75.86 + 1,796.67 = 1,872.53, and 75.86 of them is 4.05%. The same tool has gone from a coin toss to about 4 in 100 flagged adults.

This share is the positive predictive value (PPV): of the people flagged, the share who have the disease. The negative predictive value (NPV) is the share of cleared people who are free of the disease. In the study, 98 of the 134 cleared were free, 73.13%; in the district, 98.47% are.

The district's high NPV says less than it seems. Before any test, 98.00% of its adults are already free of the disease, so the tool adds less than half a percentage point, and it still clears 124 people who have the disease.

In symbols, each predictive value can be written with the counts or with Se, Sp and π:

$$PPV=\frac{TP}{TP+FP}=\frac{Se\,\pi}{Se\,\pi+(1-Sp)(1-\pi)}$$

$$NPV=\frac{TN}{TN+FN}=\frac{Sp\,(1-\pi)}{Sp\,(1-\pi)+(1-Se)\,\pi}$$

Sensitivity and specificity start from the true disease status and ask about the result. Predictive values start from the result and ask about the disease, and that reversal is why the prevalence π enters.

# Illustrative numbers for the committee's case: the company's study of 178 people
tp <- 22; fn <- 36; fp <- 22; tn <- 98; n <- tp + fn + fp + tn
pct <- function(x) formatC(100 * x, format = "f", digits = 2)
f4 <- function(x) formatC(x, format = "f", digits = 4)
cat("With the disease: ", tp + fn, "  without: ", fp + tn, "  all: ", n, "\n", sep = "")
cat("Sensitivity: ", tp, " of ", tp + fn, " = ", pct(tp / (tp + fn)), "%\n", sep = "")
cat("Specificity: ", tn, " of ", tn + fp, " = ", pct(tn / (tn + fp)), "%\n", sep = "")
cat("Accuracy: ", tp + tn, " of ", n, " = ", pct((tp + tn) / n), "%\n", sep = "")
cat("Prevalence in the study: ", tp + fn, " of ", n, " = ", pct((tp + fn) / n), "%\n", sep = "")
cat("Weighted: ", f4((tp + fn) / n), " x ", pct(tp / (tp + fn)), "% + ", f4((fp + tn) / n), " x ",
    pct(tn / (tn + fp)), "% = ", pct(tp / n), "% + ", pct(tn / n), "%\n", sep = "")
cat("PPV: ", tp, " of ", tp + fp, " flagged = ", pct(tp / (tp + fp)), "%\n", sep = "")
cat("NPV: ", tn, " of ", tn + fn, " cleared = ", pct(tn / (tn + fn)), "%\n", sep = "")

With the disease: 58  without: 120  all: 178
Sensitivity: 22 of 58 = 37.93%
Specificity: 98 of 120 = 81.67%
Accuracy: 120 of 178 = 67.42%
Prevalence in the study: 58 of 178 = 32.58%
Weighted: 0.3258 x 37.93% + 0.6742 x 81.67% = 12.36% + 55.06%
PPV: 22 of 44 flagged = 50.00%
NPV: 98 of 134 cleared = 73.13%

# Illustrative numbers for the committee's case: 10,000 adults screened once, 2% with the disease
people <- 10000; prevalence <- 0.02; se <- 22 / 58; sp <- 98 / 120
sick <- people * prevalence; well <- people - sick
tp <- sick * se; fn <- sick - tp; fp <- well * (1 - sp); tn <- well - fp
f2 <- function(x) formatC(x, format = "f", digits = 2, big.mark = ",")
cat("With the disease: ", sick, "  without: ", well, "\n", sep = "")
cat("Sick flagged, sick cleared, well flagged, well cleared:", f2(c(tp, fn, fp, tn)), "\n")
cat("Rounded to whole people:", round(c(tp, fn, fp, tn)), "\n")
cat("Flagged: ", round(tp) + round(fp), "  cleared: ", round(fn) + round(tn), "\n", sep = "")
cat("PPV: ", f2(tp), " / ", f2(tp + fp), " = ", f2(100 * tp / (tp + fp)), "%\n", sep = "")
cat("NPV: ", f2(100 * tn / (tn + fn)), "%  (no test: ", f2(100 * well / people), "%)\n", sep = "")
cat("Accuracy: ", f2(100 * se * prevalence), "% + ", f2(100 * sp * (1 - prevalence)), "% = ",
    f2(100 * (tp + tn) / people), "%\n", sep = "")

With the disease: 200  without: 9800
Sick flagged, sick cleared, well flagged, well cleared: 75.86 124.14 1,796.67 8,003.33
Rounded to whole people: 76 124 1797 8003
Flagged: 1873  cleared: 8127
PPV: 75.86 / 1,872.53 = 4.05%
NPV: 98.47%  (no test: 98.00%)
Accuracy: 0.76% + 80.03% = 80.79%

The same tool at every prevalence

Only the prevalence changed between the study and the district. The figure below holds the tool fixed and moves the prevalence from 0% to 40%.

The PPV climbs fastest at low prevalence and more slowly afterwards, from 4.05% at the district's 2% to 50.00% at the study's 32.58%, and it is still rising at 40%. The NPV falls as the disease becomes common, and accuracy drifts down in a straight line from the specificity toward the sensitivity. A PPV, an NPV or an accuracy quoted from a study belongs to that study's mix of people, and must be worked out again for the district.

One person's chance before and after the result: likelihood ratios

Read first on this site:

• Likelihood ratios and the conditional probability beneath them
• Sensitivity, specificity, likelihood ratios and ROC curves in clinical decisions

A doctor with one flagged patient needs to know how far the result should move that patient's chance, from wherever it started. The tool flags 37.93% of sick people but also 18.33% of well people. A positive result is therefore 37.93 / 18.33 = 2.069 times as common in the sick as in the well: the positive likelihood ratio, LR+. A negative result is 62.07 / 81.67 = 0.760 times as common in the sick as in the well: the negative likelihood ratio, LR−.

Likelihood ratios multiply odds, not chances. In the district the odds are 2 to 98, or 0.0204. After a positive, the odds become 0.0204 × 2.069 = 0.0422, which is a chance of 0.0422 / (1 + 0.0422) = 4.05%, the PPV again. After a negative, 0.0204 × 0.760 = 0.0155, a chance of 1.53%: the 124 missed among the 8,127 cleared.

$$LR^{+}=\frac{Se}{1-Sp}$$

$$\text{odds after the result}=\text{odds before}\times LR$$

The same rule recovers the study: its odds of 58 to 120 (0.4833) times 2.069 give 1.000, even odds, the study's PPV of 50.00%. For a positive to reach even odds in the district, the ratio would have to be about 49.

Ratios above 10, or below one tenth, change a doctor's judgement a lot; ratios near 1 rarely do [1]. This tool's 2.069 and 0.760 are both near 1, so it is a weak test. A positive raises the chance only from 2% to 4.05%, so most flagged villagers are still well, and a negative lowers it only from 2% to 1.53%.

# Illustrative numbers for the committee's case: how far one result moves one person's chance
se <- 22 / 58; sp <- 98 / 120; lrp <- se / (1 - sp); lrn <- (1 - se) / sp; pre <- 0.02 / 0.98
f <- function(x, d = 3) formatC(x, format = "f", digits = d); p <- function(o) f(100*o / (1+o), 2)
cat("Flagged if sick: ", f(100 * se, 2), "%  if well: ", f(100 * (1 - sp), 2), "%; cleared, sick: ",
    f(100 * (1 - se), 2), "%  well: ", f(100 * sp, 2), "%\n", sep = "")
cat("Positive LR: ", f(lrp), "  negative LR: ", f(lrn), "\n", sep = "")
cat("District odds 2 to 98 = ", f(pre, 4), "; after a positive x ", f(lrp), " = ", f(pre * lrp, 4),
    " -> ", p(pre * lrp), "%\n", sep = "")
cat("After a negative: ", f(pre, 4), " x ", f(lrn), " = ", f(pre * lrn, 4), " -> ", p(pre * lrn),
    "%\n", sep = "")
cat("Study odds: 58 to 120 = ", f(58 / 120, 4), " -> after a positive ", f(58 / 120 * lrp), "\n",
    sep = "")
cat("LR a positive would need for even odds in the district: ", f(1 / pre, 0), "\n", sep = "")

Flagged if sick: 37.93%  if well: 18.33%; cleared, sick: 62.07%  well: 81.67%
Positive LR: 2.069  negative LR: 0.760
District odds 2 to 98 = 0.0204; after a positive x 2.069 = 0.0422 -> 4.05%
After a negative: 0.0204 x 0.760 = 0.0155 -> 1.53%
Study odds: 58 to 120 = 0.4833 -> after a positive 1.000
LR a positive would need for even odds in the district: 49

Following the flagged into the budget: cost per case detected

The committee pays for two things: a screening test for everyone and a confirmation test for everyone flagged. Use illustrative prices of 200 THB per screening test and 2,000 THB per confirmation. Money is paid per whole person, so this section counts 76 cases and 1,797 false alarms.

Testing costs 10,000 × 200 = 2,000,000 THB. Confirming the 1,873 flagged adults costs 1,873 × 2,000 = 3,746,000 THB. The round costs 5,746,000 THB in total and finds 76 cases: 5,746,000 / 76 = 75,605 THB per case found.

That figure is the cost per case detected: the programme's spending divided by the true cases it finds. It is a useful first number, but it says nothing yet about whether finding those cases helps anyone.

Look at where the money goes. Confirming the 1,797 false alarms costs 3,594,000 THB, 62.55% of the total. The remaining 152,000 THB confirms the 76 true cases (76 × 2,000), so testing, false alarms and true cases add to 5,746,000 THB.

Each case found comes with 1,797 / 76 = 23.64 false alarms, each one a worried villager and a trip to the hospital. Meanwhile 124 people with the disease were told they were clear.

# Illustrative prices for the committee's case: one screening round of 10,000 adults
people <- 10000; test <- 200; confirm <- 2000; tp <- 76; fp <- 1797
testing <- people * test; confirming <- (tp + fp) * confirm; total <- testing + confirming
thb <- function(x) formatC(x, format = "d", big.mark = ",")
cat("Testing: ", thb(people), " x ", test, " = ", thb(testing), " THB\n", sep = "")
cat("Confirming: ", thb(tp + fp), " x ", thb(confirm), " = ", thb(confirming), " THB\n", sep = "")
cat("Total: ", thb(total), " THB\n", sep = "")
cat("Per case found: ", thb(total), " / ", tp, " = ", thb(round(total / tp)), " THB\n", sep = "")
cat("Confirming false positives: ", thb(fp * confirm), " THB = ",
    formatC(100 * fp * confirm / total, format = "f", digits = 2), "% of the total\n", sep = "")
cat("Confirming cases: ", tp, " x ", thb(confirm), " = ", thb(tp * confirm), " THB\n", sep = "")
cat("False positives per true case: ", thb(fp), " / ", tp, " = ",
    formatC(fp / tp, format = "f", digits = 2), "\n", sep = "")

Testing: 10,000 x 200 = 2,000,000 THB
Confirming: 1,873 x 2,000 = 3,746,000 THB
Total: 5,746,000 THB
Per case found: 5,746,000 / 76 = 75,605 THB
Confirming false positives: 3,594,000 THB = 62.55% of the total
Confirming cases: 76 x 2,000 = 152,000 THB
False positives per true case: 1,797 / 76 = 23.64

Can the district trust 37.93% and 81.67%?

Read first on this site:

• How to judge the risk of bias in an accuracy study with QUADAS-2
• The kinds of bias in diagnostic accuracy studies, with examples
• What an accuracy study must report: STARD 2015

Every number so far assumed that the study's sensitivity and specificity hold in the district. Two things can make them wrong: random error, because the study was small, and bias, because of who it recruited (Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect separates the two).

Only 58 people in the study had the disease, so 22 of 58 is a rough estimate. Its 95% confidence interval is wide: the data are compatible with a true sensitivity anywhere from 26.56% to 50.80%. For the specificity, from 120 people, the interval runs from 73.80% to 87.57%. Both intervals use the Wilson method [2], made for shares, which stays between 0% and 100% even in small groups.

As a rough best case, if both sat at their most favourable ends, the district's PPV would be 7.70%; at their least favourable ends, 2.03%. Either way, most flagged adults in the district would be false alarms. The committee's model should carry both intervals, and this range of PPVs, into its uncertainty analysis instead of one point value.

Bias is the bigger worry. The study took 58 patients already diagnosed at a hospital and 120 healthy volunteers, so people with and without the disease entered through two different doors, a two-gate design. Hospital patients tend to have advanced, obvious disease, and healthy volunteers have nothing that resembles it, so the tool's job was easier than in a village.

A sensitivity or specificity pushed away from the real setting by the study's mix of people has spectrum bias. In a screening round the disease is usually earlier and smaller, so the district's true sensitivity is likely lower than 37.93%. A model that keeps 37.93% counts too many cases found and too few missed, so a lower sensitivity belongs in its scenarios.

A study fit for the committee would test consecutive adults like the district's and confirm everyone, flagged or not, with the same test. QUADAS-2 [3] judges these risks of bias, and STARD 2015 [4] lists what such a study must report.

Purpose matters in the same way. A tool built to help diagnose people who already have symptoms meets a sicker mix, and a higher prevalence, than one used to screen people who feel well. A tool checked for one job must be checked again before it is used for the other.

# Illustrative numbers for the committee's case: how sure 22 of 58 and 98 of 120 can be
pct <- function(x) formatC(100 * x, format = "f", digits = 2)
wilson <- function(k, m) prop.test(k, m, correct = FALSE)$conf.int
se <- wilson(22, 58); sp <- wilson(98, 120)
cat("Sensitivity: ", pct(22 / 58), "% (95% CI ", pct(se[1]), "% to ", pct(se[2]), "%)\n", sep = "")
cat("Specificity: ", pct(98 / 120), "% (95% CI ", pct(sp[1]), "% to ", pct(sp[2]), "%)\n", sep = "")
ppv <- function(a, b) 0.02 * a / (0.02 * a + 0.98 * (1 - b))
cat("District PPV with both at their best ends: ", pct(ppv(se[2], sp[2])), "%\n", sep = "")
cat("District PPV with both at their worst ends: ", pct(ppv(se[1], sp[1])), "%\n", sep = "")

Sensitivity: 37.93% (95% CI 26.56% to 50.80%)
Specificity: 81.67% (95% CI 73.80% to 87.57%)
District PPV with both at their best ends: 7.70%
District PPV with both at their worst ends: 2.03%

Finding a case is not the same as helping a person

Suppose the confirmation test is perfect and all 76 cases found are treated. The committee still does not know whether anyone lives longer or better. Three effects can make a screening programme look good even when it helps no one.

First, the clock starts earlier. Take a villager whose disease would cause symptoms at 62 and death at 70, whatever is done. Without screening they live 8 years after diagnosis; screened at 58, they live 12, yet they die at 70 either way. The 4-year head start is the lead time, and survival that looks longer only because the clock started earlier is lead-time bias.

Second, a single screen finds the slow kind. Suppose the disease has a slow form that can be found for 4 years before symptoms and a fast form that can be found for 1 year, with equal numbers of each starting every year. The screen catches the slow form 4 times as often: 80% of the cases it finds are slow, against 50% of all new cases. This tilt toward slow disease, which would have done better anyway, is length-time bias.

Third, some disease never mattered. Suppose a quarter of the 76 cases found would never have caused symptoms in the person's lifetime: 19 people would be labelled and treated with no chance of benefit. Finding such disease is overdiagnosis. It is not a false positive, because the disease is real, but finding it cannot help.

All three flatter the survival of screen-found cases. Only a randomized screening trial settles the question: it invites some people and not others by chance and counts deaths, or the outcomes people care about, in everyone invited and everyone not (Beyond the Evidence Pyramid: GRADE Certainty and Choosing Effectiveness Inputs for Economic Models rates such evidence).

# Illustrative numbers for the committee's case: finding the disease earlier is not helping
screened_at <- 58; symptoms_at <- 62; death_at <- 70
cat("Years from diagnosis to death: ", death_at - symptoms_at, " without screening, ",
    death_at - screened_at, " with\n", sep = "")
cat("Age at death either way: ", death_at, "  lead time: ", symptoms_at - screened_at, " years\n",
    sep = "")
slow <- 4; fast <- 1
cat("Slow form among the cases one screen finds: ", 100 * slow / (slow + fast),
    "% (", 100 * 1 / 2, "% of new cases)\n", sep = "")
found <- 76; harmless <- 0.25
cat("Cases found that would never have caused harm: ", found * harmless, " of ", found, "\n",
    sep = "")

Years from diagnosis to death: 8 without screening, 12 with
Age at death either way: 70  lead time: 4 years
Slow form among the cases one screen finds: 80% (50% of new cases)
Cases found that would never have caused harm: 19 of 76

How many false alarms is one case worth? Threshold and net benefit

Read first on this site:

• Calibration and decision curve analysis for prediction models
• How to judge a prediction model, including whether it is useful in practice

The committee still has to weigh 76 cases found against 1,797 false alarms. That weighing is a value judgement, and it can be stated before anyone looks at the tool.

The committee agrees on a rule: send a person to confirmation if their chance of having the disease is at least 5%. At that chance, 5 people with the disease stand against 95 without, so the committee accepts up to 95 / 5 = 19 false alarms for each case found. Such a cut-off chance, at or above which acting is worth its harms, is called the threshold probability.

The tool produces 23.64 false alarms per case found (1,797 / 76 in whole people, 23.68 before rounding), more than the 19 the committee accepts. Put another way, a flagged villager's chance of disease, the PPV of 4.05%, is below the 5% threshold. By the committee's own rule, sending flagged people to confirmation does more harm than good.

One number can carry this: count the true cases found, then subtract the false alarms, each counted as 1/19 of a case. Per 10,000 adults: 75.86 − 1,796.67 / 19 = 75.86 − 94.56 = −18.7. Confirming everyone gives 200 − 515.79 = −315.8, and confirming no one gives 0. This count is the net benefit.

So the tool beats confirming everyone but loses to doing nothing. If the committee would accept 49 false alarms per case (98 / 2 = 49, a 2% threshold), the tool's net benefit would be 75.86 − 36.67 = 39.2 per 10,000, and screening would win.

One caution on all of these counts: they are generous to the tool. They treat every case found as a case helped, and the previous section showed that is not guaranteed, so the tool's true net benefit can only be lower.

Repeated at every threshold, these sums draw one line each for the tool, for confirming everyone and for confirming no one; comparing the lines is decision curve analysis [5]. The tool's line is positive at 2%, zero at 4.05%, its PPV, and negative at 5%. Below 1.53%, the chance left after a negative result, confirming everyone scores highest; between 1.53% and 4.05% the tool is best; above 4.05% doing nothing wins.

With N adults screened and the threshold p_t, the net benefit per person is:

$$NB=\frac{TP}{N}-\frac{FP}{N}\times\frac{p_t}{1-p_t}$$

At p_t = 5% the weight p_t/(1 − p_t) is 5/95, the 1/19 used above, and the formula gives −18.7 / 10,000 = −0.00187 per person; we quote it per 10,000 adults because that is easier to read. Net benefit counts cases, not baht. The economic version puts a money value on the health gained and subtracts the costs, and later articles in this series build it. There, a price per unit of health plays the role the threshold plays here: both say, before the data, how much one good outcome is worth.

# Illustrative numbers for the committee's case: net benefit per 10,000 adults screened
people <- 10000; sick <- 200; well <- people - sick
tp <- sick * 22 / 58; fp <- well * (1 - 98 / 120)
f <- function(x, d = 2) formatC(x, format = "f", digits = d, big.mark = ",")
cat("False alarms per case found before rounding: ", f(fp), " / ", f(tp), " = ", f(fp / tp),
    "\n", sep = "")
cat("Threshold 5%: one false positive counts as 1/", 0.95 / 0.05, " of a true case\n", sep = "")
cat("Tool: ", f(tp), " - ", f(fp), " / 19 = ", f(tp), " - ", f(fp / 19), " = ",
    f(tp - fp / 19, 1), "; per person ", f((tp - fp / 19) / people, 5), "\n", sep = "")
cat("Confirm everyone: ", sick, " - ", f(well / 19), " = ", f(sick - well / 19, 1),
    "; confirm no one: ", f(0, 1), "\n", sep = "")
cat("Threshold 2% (1/", 0.98 / 0.02, "): tool ", f(tp), " - ", f(fp / 49), " = ",
    f(tp - fp / 49, 1), "; confirm everyone ", f(sick - well / 49, 1), "\n", sep = "")

False alarms per case found before rounding: 1,796.67 / 75.86 = 23.68
Threshold 5%: one false positive counts as 1/19 of a true case
Tool: 75.86 - 1,796.67 / 19 = 75.86 - 94.56 = -18.7; per person -0.00187
Confirm everyone: 200 - 515.79 = -315.8; confirm no one: 0.0
Threshold 2% (1/49): tool 75.86 - 36.67 = 39.2; confirm everyone 0.0

What the committee's economic model still needs

Read first on this site:

• Reading a calibration plot for a prediction model
• How an economic evaluation shows what its uncertain inputs do to the result

A cost of 75,605 THB per case found is an intermediate result. An economic evaluation of the programme compares it with what the district does now. That comparison, the extra cost per extra unit of health against the district's current practice, is the next step in the series; the series list at the end of this article shows where it sits. To make it, the model follows each group of people to the outcomes that matter [6]:

• the 76 cases found: how much earlier treatment changes their health, taken from randomized screening trials, not from survival after diagnosis;
• the 1,797 false alarms: the cost of confirming them and the worry it causes;
• the 124 missed cases, falsely reassured, and the cases found that never needed finding;
• how often to screen: the first round finds disease that has built up over years, while later rounds find mostly new cases, so their prevalence, and their PPV, will be lower.

Of all the numbers in this article, the model takes only a few as inputs: the sensitivity and specificity with their intervals, the prevalence expected in each round, and the prices. The predictive values, the counts of cases found and false alarms, and the cost per case are outputs. They are worked out again for each round and each district, never copied from a study.

An AI tool needs one more check. Many such tools give each person a predicted chance of disease, not just a flag. Calibration is the agreement between those chances and reality: among people told they have a 1 in 10 chance, about 1 in 10 should turn out to have the disease.

A tool can rank people well and still be badly calibrated in a district whose prevalence differs from the data it learned from. So the local prevalence, the calibration and the cut-off for a flag must be checked in the district before use.

The series, in learning order

1. Health Economics Series Guide: Read the Evidence Before the Costs
2. Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models
3. Design, Model and Estimand: What Each Study Design Can Actually Estimate
4. Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers
5. The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs
6. Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect
7. Beyond the Evidence Pyramid: GRADE Certainty and Choosing Effectiveness Inputs for Economic Models
8. From Sensitivity to Cost per Case Detected: Predictive Values, Prevalence and the Economics of Screening (this article)
9. Whose Cost, in Which Year? Perspective, Opportunity Cost, Inflation and Discounting
10. QALY, DALY and the Utilities Behind Them: What Is Actually Being Measured
11. ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY
12. Decision Trees, Markov Cohorts and Probabilistic Sensitivity Analysis: How a Model Turns Evidence into a Decision
13. Cost-Effective Is Not Affordable, and Efficient Is Not Fair: Budget Impact and Equity
14. Parameter Sourcing, Reference Cost Lists and Utility Catalogues When Local Data Are Sparse
15. Early Health Technology Assessment, Headroom Pricing and Benefit-Package Decisions
16. Communicating Health Economic Evidence: From Model Outputs to Policy Briefs

Related in the wiki: Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models · Beyond the Evidence Pyramid: GRADE Certainty and Choosing Effectiveness Inputs for Economic Models · ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY · Decision Trees, Markov Cohorts and Probabilistic Sensitivity Analysis: How a Model Turns Evidence into a Decision · Diagnostic Indices: Sensitivity, Specificity, Predictive Values, and Beyond · Why Sensitivity Is Not Kappa: Performance vs Agreement