Modes of convergence, WLLN, SLLN, CLT

The diagram

   L^p  ──────▶  probability  ──────▶  distribution
                     ▲
almost sure ─────────┘

No other arrows hold in general. Partial converses:

• convergence in probability ⇒ an a.s. convergent subsequence;
• convergence in distribution to a constant ⇒ convergence in probability;
• probability + uniform integrability $\Rightarrow L^{1}$; $- \sum P($|$X_{n} - X$| $> \varepsilon) < \infty \Rightarrow a.s. ($Borel–Cantelli).

The counterexamples

Laws of large numbers

• WLLN (in probability) needs finite mean; SLLN (a.s.) also holds with finite mean (Kolmogorov).
• Fails without a mean: the Cauchy sample mean has the same Cauchy distribution for every n.

Central limit theorem

$\sqrt{n}({\bar{X}} - \mu)/\sigma \to N(0,1)$ whenever $\sigma^{2} < \infty$. Consequences examined: $- P(S_{n} \le n\mu) \to$ ½, so intervals with the mean as an endpoint have limiting probability ½, while intervals containing $\mu$ in the interior have probability → 1.

• Delta method: $\sqrt{n}(g({\bar{X}}) - g(\mu)) \to N(0, \sigma^{2}g'(\mu)^{2})$.
• Slutsky: $X_{n} \Rightarrow X$ and $Y_{n} \to c$ in probability $\Rightarrow X_{n} + Y_{n} \Rightarrow X + c, X_{n}Y_{n} \Rightarrow$ cX.

Key takeaways

• a.s. and L^p both imply probability, which implies distribution; nothing else.
• Typewriter kills "probability ⇒ a.s."; n·$1_{(0,1/n)}$ kills "$a.s. \Rightarrow L^{1}$".
• CLT at the boundary gives ½ — a favourite Part-C trick.