Modes of convergence, WLLN, SLLN, CLT

In one sentence

Four modes of convergence sit in a strict hierarchy, the laws of large numbers give two of them under an integrability hypothesis, and the central limit theorem needs finite variance rather than normality.

Why the exam asks it

Because the modes are easy to state and easy to confuse, each reverse implication has a small standard counterexample, and the CLT's hypothesis is routinely misremembered as "the data must be normal".

The idea

The four modes

• Almost surely: $P(X_n\to X)=1$.
• In probability: $P(|X_n-X|>\varepsilon)\to0$ for every $\varepsilon$.
• In $L^p$: $E|X_n-X|^p\to0$.
• In distribution: $F_n(x)\to F(x)$ at every continuity point of $F$.

$$\text{a.s.} \Rightarrow \text{in probability} \Rightarrow \text{in distribution}, \qquad L^p \Rightarrow \text{in probability}.$$

Almost sure and $L^p$ imply neither one another. Every reverse arrow fails:

In probability, not almost surely. The typewriter sequence: indicators of $[k/2^n,(k+1)/2^n]$ marching across $[0,1]$. The intervals shrink so $P(X_n\ne0)\to0$, but every point is covered infinitely often, so $X_n(\omega)$ does not converge for any $\omega$. This is the same sequence that separates $L^p$ convergence from pointwise convergence in the Lebesgue chapter — one example, two topics.

In probability, not in $L^1$. $X_n = n$ on $(0,1/n)$ and $0$ elsewhere. It tends to $0$ in probability and $E[X_n]=1$ always. Convergence in probability controls where the mass is, not how heavy it is.

In distribution, not in probability. Let $X$ be symmetric and $X_n=-X$ for every $n$. Then $X_n$ has the same distribution as $X$, so it converges in distribution to $X$ — while $|X_n-X|=2|X|$ never gets small. Distributional convergence says nothing about the variables being close, only their laws.

The one reversal that holds: convergence in distribution to a constant implies convergence in probability. That exception is worth knowing, since it is what makes Slutsky's theorem usable.

The laws of large numbers

Both concern $\bar X_n = \frac1n\sum X_i$ for iid $X_i$ with mean $\mu$.

• Weak law: $\bar X_n \to \mu$ in probability. Chebyshev proves it when the variance is finite, since $\operatorname{Var}(\bar X_n)=\sigma^2/n\to0$; it holds under $E|X|<\infty$ alone.
• Strong law: $\bar X_n\to\mu$ almost surely, under $E|X|<\infty$.

The strong law implies the weak one. The hypothesis in both is integrability, and Cauchy fails it: with no mean there is nothing to converge to, and in fact $\bar X_n$ is itself Cauchy for every $n$ — averaging achieves nothing at all.

The central limit theorem

$$\frac{\sqrt n(\bar X_n-\mu)}{\sigma} \xrightarrow{d} N(0,1),$$

for iid $X_i$ with finite variance. No normality is required of the $X_i$ — that is the whole point, and it is why the normal appears everywhere.

The scaling is $\sqrt n$: the sum has standard deviation $\sigma\sqrt n$, so dividing by $n$ over-shrinks and dividing by $\sqrt n$ is exactly right. Getting that exponent wrong is the standard slip.

Finite variance is the hypothesis. Heavy-tailed variables with infinite variance converge to stable laws instead, and Cauchy is again the extreme case.

The tools

• Slutsky: if $X_n\xrightarrow{d}X$ and $Y_n\xrightarrow{p}c$, then $X_n+Y_n\xrightarrow{d}X+c$ and $X_nY_n\xrightarrow{d}cX$. The second sequence must go to a constant.
• Continuous mapping: $g$ continuous preserves all three modes.
• Delta method: $\sqrt n(\bar X-\mu)\xrightarrow{d}N(0,\sigma^2)$ gives $\sqrt n(g(\bar X)-g(\mu))\xrightarrow{d}N(0,g'(\mu)^2\sigma^2)$ — the CLT pushed through a smooth function, and the standard route to the variance of an estimator.

Where intuition breaks

"Convergence in probability implies almost sure convergence." Typewriter sequence. It implies a subsequence converges almost surely, which is the correct weakening.

"The CLT requires the data to be normal." It requires finite variance. Normality of the $X_i$ would make the result trivial.

"The law of large numbers always applies." It needs $E|X|<\infty$. The Cauchy sample mean is Cauchy for every $n$.

"Convergence in distribution means the variables get close." $X_n=-X$ converges in distribution to $X$ and stays as far away as ever. Only the laws converge.

"Slutsky works for any two convergent sequences." The second must converge in probability to a constant. Two sequences converging in distribution cannot be combined.

"Scaling by $n$ in the CLT." By $\sqrt n$. Dividing by $n$ gives the law of large numbers and a degenerate limit.

"Convergence in $L^2$ implies almost sure convergence." Neither implies the other. Both imply convergence in probability.

The exam's angle

1. Identify the mode being claimed before anything else — half the options here are the right theorem with the wrong mode.
2. Check integrability for a law of large numbers, finite variance for the CLT. That is the difference between the two hypotheses and it is what the option usually drops.
3. Keep the three counterexamples ready: typewriter, the tall spike, and $X_n=-X$.
4. For a function of an estimator, use the delta method and quote $g'(\mu)^2\sigma^2$.
5. Remember the constant exception — in distribution to a constant upgrades to in probability, and that is what licenses Slutsky.

The night before

• a.s. $\Rightarrow$ in probability $\Rightarrow$ in distribution; $L^p\Rightarrow$ in probability. a.s. and $L^p$ imply each other in neither direction.
• Typewriter: in probability, not a.s. Spike $n$ on $(0,1/n)$: in probability, not $L^1$. $X_n=-X$: in distribution, not in probability.
• In distribution to a constant $\Rightarrow$ in probability.
• WLLN in probability, SLLN almost surely, both under $E|X|<\infty$. Cauchy fails: its sample mean is Cauchy.
• CLT: $\sqrt n(\bar X-\mu)/\sigma\to N(0,1)$, needing finite variance, not normality.
• Slutsky needs the second limit to be a constant. Continuous mapping preserves all modes.
• Delta method: $\sqrt n(g(\bar X)-g(\mu))\to N(0, g'(\mu)^2\sigma^2)$.