Axioms, conditional probability, independence, Bayes

In one sentence

Probability is a measure of total mass one, conditioning renormalises it, and the two things that catch people are that pairwise independence is weaker than mutual, and that a positive test on a rare condition usually means nothing.

Why the exam asks it

Because the base-rate error is universal and has a definite numerical answer, and because "independent" has two inequivalent meanings for three or more events. Both are testable in one line.

The idea

Kolmogorov's axioms: $P(\Omega)=1$, $P(A)\ge0$, and countable additivity for disjoint events. Everything follows, including $P(A^c)=1-P(A)$ and inclusion–exclusion.

Conditional probability: $P(A\mid B) = \dfrac{P(A\cap B)}{P(B)}$ for $P(B)>0$. This is a genuine probability measure on $B$ — the mass is renormalised, not moved.

Law of total probability: for a partition $\{B_i\}$, $P(A) = \sum_i P(A\mid B_i)P(B_i)$.

Bayes:

$$P(B_i \mid A) = \frac{P(A\mid B_i)P(B_i)}{\sum_j P(A\mid B_j)P(B_j)}.$$

The base rate

A disease affects $1$ in $1000$. A test is $99\%$ accurate in both directions. You test positive. The probability you are ill is

$$\frac{0.001 \times 0.99}{0.001\times0.99 + 0.999\times0.01} = \frac{99}{1098} = \frac{11}{122} \approx 9\%.$$

The counts make it obvious: among $100{,}000$ people, $100$ are ill and $99$ of them test positive; $99{,}900$ are well and $999$ of them test positive anyway. Ten times as many false positives as true ones, because the well group is a thousand times larger.

The error is ignoring $P(B_i)$ — treating $P(A\mid B)$ as if it were $P(B\mid A)$. Those are different numbers whenever the base rates differ, which is nearly always.

Independence, and its two versions

$A$ and $B$ are independent if $P(A\cap B)=P(A)P(B)$.

For three or more events, mutual independence requires the product rule for every subset — not just for pairs. Pairwise independence is strictly weaker.

The standard witness: toss two fair coins. Let $A$ = first is heads, $B$ = second is heads, $C$ = the two agree. Each has probability $\tfrac12$, and each pair is independent: $P(A\cap B)=P(A\cap C)=P(B\cap C)=\tfrac14$. But

$$P(A\cap B\cap C) = \tfrac14 \ne \tfrac18 = P(A)P(B)P(C),$$

because $A$ and $B$ together already determine $C$. Pairwise independent, not mutually so.

Independent is not disjoint. Disjoint events with positive probability are dependent — knowing one occurred tells you the other did not. $P(A\cap B)=0 \ne P(A)P(B)$. The words feel similar and mean nearly opposite things.

Where intuition breaks

"A 99% accurate positive means 99% chance of disease." About $9\%$ here. The base rate dominates when the condition is rare, and this is the single most examined error in probability.

"$P(A\mid B) = P(B\mid A)$." Equal only when $P(A)=P(B)$. Bayes is precisely the correction factor between them.

"Pairwise independent means independent." The two-coin example. Mutual independence needs every subset to factor.

"Independent events are disjoint." Nearly the opposite. Disjoint events with positive probability are strongly dependent.

"$P(A\cup B) = P(A)+P(B)$." Only for disjoint events. Otherwise subtract $P(A\cap B)$.

"Conditioning on a null event is fine if we take a limit." $P(A\mid B)$ is undefined for $P(B)=0$, and conditioning on a measure-zero event needs the conditional-density machinery — different limits give different answers, which is the Borel–Kolmogorov paradox.

"Independence is transitive." $A$ independent of $B$ and $B$ of $C$ says nothing about $A$ and $C$.

The exam's angle

1. Convert to counts. Bayes problems become arithmetic when written as frequencies out of $100{,}000$, and the base-rate effect becomes visible rather than surprising.
2. Identify the partition before applying total probability — the denominator is a sum over it.
3. For three events, check every subset when asked about independence, and expect pairwise to be the trap.
4. Watch for "disjoint" offered as evidence of independence.
5. Ask which conditional is given and which is wanted. Most wrong options swap them.

The night before

• $P(A\mid B)=P(A\cap B)/P(B)$; total probability sums over a partition; Bayes reverses the conditional.
• Rare disease, $99\%$ test: $P(\text{ill}\mid+) = 11/122 \approx 9\%$. Ten false positives per true one.
• Pairwise $\ne$ mutual independence: two coins and "they agree" is pairwise independent with $P(ABC)=1/4\ne1/8$.
• Disjoint $\ne$ independent — disjoint with positive probability means dependent.
• $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.
• Independence is not transitive.