Axioms, conditional probability, independence, Bayes

Basics

P(A ∪ B) = P(A) + P(B) − P(A∩B); inclusion–exclusion for more sets. **Boole$**: P(\cup{}A_{i}) \le \sum{}P(A_{i}). **$Bonferroni$**: P(\cap{}A_{i}) \ge 1 - \sum{}P(A_{i}$ᶜ).

Conditioning

P(A|B) = P(A∩B)/P(B). **Bayes$**: P(A_{i}$|B) = P(B|$A_{i})P(A_{i})/\sum{}P(B$|$A_{j})P(A_{j})$.

Symmetry facts (the recurring Part-C pattern):

Statement	Symmetric in A, B?
P(A|B) = 0 ⇔ P(B|A) = 0	✓ (both say P(A∩B) = 0)
P(A|B) > P(A) ⇔ P(B|A) > P(B)	✓ (both say P(A∩B) > P(A)P(B))
P(A|B) = 1 ⇒ P(B|A) = 1	✗ (A ⊇ B does not give B ⊇ A)
P(A|B) > P(B) ⇒ P(B|A) > P(A)	✗

Independence

• A ⫫ B ⇔ P(A∩B) = P(A)P(B). Then Aᶜ ⫫ B too.
• Pairwise independence does not imply mutual independence: two fair coins with A = first head, B = second head, C = "same face" are pairwise independent but not mutually.
• Independent events with positive probability are never disjoint.

Standard traps

• Mutually exclusive ≠ independent (they are opposites unless one has probability 0).
• Conditional independence given C does not imply independence, and vice versa (Simpson's paradox territory).

Key takeaways

• Convert every conditional statement to P(A∩B) vs P(A)P(B) and symmetry becomes obvious.
• Pairwise ⇏ mutual independence.
• Disjoint and independent are almost opposite notions.