Bayes' theorem

Theorem — Bayes

Let $A$ and $B$ be two events with $P(A), P(B) > 0$. Then: $P(B\mid A) = \frac{P(A\mid B)\cdot P(B)}{P(A)}.$ If moreover $\{B_1,\ldots,B_n\}$ is a partition of $\Omega$, for each $B_j$: $P(B_j\mid A) = \frac{P(A\mid B_j)\cdot P(B_j)}{\sum_i P(A\mid B_i)\cdot P(B_i)}.$

Proof

From the product formula $P(A\cap B) = P(A\mid B)\cdot P(B)$ and $P(A\cap B) = P(B\mid A)\cdot P(A)$, equating the right-hand sides gives the first formula. The second is the first with the total probability $P(A) = \sum_i P(A\mid B_i)P(B_i)$ in the denominator. ∎

Remark — The intuition behind Bayes

Bayes’ theorem lets us invert a conditional probability: from $P(A\mid B)$ to $P(B\mid A)$. It is the tool for going from a “cause” ($B$) to an “effect” ($A$) and then asking, given the effect, what is the probability that $B$ was the cause. It is at the heart of probabilistic reasoning in medicine, justice, diagnostics, and machine learning.

Example — Diagnostic test, completed

A rare disease affects $0,1\%$ of the population. A diagnostic test has sensitivity $99\%$ ($P(T^+\mid M)=0,99$) and specificity $98\%$ ($P(T^+\mid \overline{M})=0,02$). A person tests positive. What is the probability that they are actually ill?

We have already computed $P(T^+) \approx 0,02097$ (the sum of the two paths). Bayes: $P(M\mid T^+) = \frac{P(T^+\mid M)P(M)}{P(T^+)} = \frac{0,99\cdot 0,001}{0,02097} \approx 0,0472 \approx \mathbf{4,7\%}.$ Moral: even though the test is very accurate, the posterior probability of being ill given a positive test is only $4,7\%$. The reason is that the disease is so rare that the false positives (healthy people with a positive test) far outnumber the true positives. This is the famous “base rate fallacy” in the interpretation of diagnostic tests.

Links

Topics: Probability
Concepts: Conditional probability · Total probability · Bayes’ theorem
Methods: Bayes · Conditional prob · Total prob
Skills: Calculating probability · Using formulae
People: Thomas Bayes