Sensitivity, specificity, precision

A hugely important application of conditional probability are diagnostic tests (medical, anti-doping, quality, etc.). A test is never perfect: it can give false positives (an alarm without illness) or false negatives (an illness not detected).

Let us define:

• $A$ = “the subject is actually ill”;
• $B$ = “the test gives a positive result”.

The four possible combinations are:

	$A$ (ill)	$\overline{A}$ (healthy)
$B$ (test $+$)	true positive ($TP$)	false positive ($FP$)
$\overline{B}$ (test $-$)	false negative ($FN$)	true negative ($TN$)

From this table three important quantities are defined.

Definition

• Sensitivity (or recall) $= \dfrac{TP}{TP+FN} = P(B\mid A)$: the probability of testing positive given that one is ill. A sensitive test “does not let the ill slip through”.
• Specificity $= \dfrac{TN}{TN+FP} = P(\overline{B}\mid \overline{A})$: the probability of testing negative given that one is healthy. A specific test “does not raise false alarms”.
• Precision (or positive predictive value) $= \dfrac{TP}{TP+FP} = P(A\mid B)$: if the test is positive, what is the probability of actually being ill? It is the most useful quantity for the patient, and the question that Bayes’ theorem answers.

Caution — The paradox of rare tests

A test with high sensitivity and high specificity can nonetheless have low precision if the illness is rare. This is the famous “paradox of rare tests”: if only $1$ person in $10\,000$ is ill, even a test with $99\%$ sensitivity and $99\%$ specificity produces a number of false positives comparable to the number of true positives, and $P(A\mid B)$ turns out to be much lower than one would expect. It is the same mechanism as tests for low-prevalence illnesses: with an illness at $0{,}1\%$, sensitivity $99\%$ and specificity $98\%$, the precision was only $4{,}7\%$.

Links

Topics: Probability
Concepts: Precision · Conditional probability · Sensitivity · Specificity · Bayes’ theorem
Skills: Probability calculation