Random variables, distributions, moments, MGF

In one sentence

A distribution is determined by its CDF, the MGF determines it too when it exists near zero, and neither the mean nor the MGF is guaranteed to exist at all.

Why the exam asks it

Because "the expectation" is written as though it always exists, and the Cauchy distribution is the standard reminder that it need not. And because the MGF's determining property carries a hypothesis that options drop.

The idea

The CDF $F(x)=P(X\le x)$ is non-decreasing, right-continuous, with limits $0$ and $1$. It always exists and always determines the distribution — unlike everything else here.

$$E[X]=\int x\,dF, \qquad \operatorname{Var}(X)=E[X^2]-(E[X])^2 .$$

Existence is a real condition. $E[X]$ exists when $E|X|<\infty$. The Cauchy density $\frac{1}{\pi(1+x^2)}$ integrates to $1$ and has $\int|x|f = \infty$, so it has no mean — and therefore no variance and no MGF. The sample mean of Cauchy variables is itself Cauchy, so it does not settle down however many you take, which is why the law of large numbers has an integrability hypothesis.

Generating functions

$$M(t)=E[e^{tX}], \qquad \phi(t)=E[e^{itX}].$$

• $M$ may fail to exist for any $t\ne0$ — Cauchy again, and the lognormal has all moments finite with $M(t)=\infty$ for every $t>0$.
• $\phi$ always exists, since $|e^{itX}|=1$. That is why characteristic functions are the tool of choice for limit theorems.
• If $M$ exists on an open interval containing $0$, it determines the distribution and $M^{(n)}(0)=E[X^n]$.

Moments do not determine a distribution in general. The lognormal is the standard witness: an explicit family of distinct distributions shares every moment with it. So "same moments, therefore same distribution" is false, and the correct statement requires the MGF to exist near zero.

The inequalities

• Markov: $P(X\ge a)\le E[X]/a$ for $X\ge0$.
• Chebyshev: $P(|X-\mu|\ge k\sigma)\le1/k^2$. Distribution-free, and therefore weak — for the normal at $k=2$ the true tail is $0.046$ against Chebyshev's $0.25$.
• Jensen: $E[g(X)]\ge g(E[X])$ for convex $g$. Hence $E[X^2]\ge(E[X])^2$, which is variance being non-negative, and $E[1/X]\ge1/E[X]$ for positive $X$.
• Cauchy–Schwarz: $E[XY]^2\le E[X^2]E[Y^2]$, giving $|\rho|\le1$.

Where intuition breaks

"Every random variable has a mean." Cauchy. And with no mean there is no variance, no MGF, and no law of large numbers.

"The MGF always exists." It needs $E[e^{tX}]<\infty$ near $0$. Heavy tails destroy it while leaving the characteristic function intact.

"Same moments means same distribution." False in general — the lognormal counterexample. True when the MGF exists in a neighbourhood of zero, which is the hypothesis that matters.

"$E[g(X)]=g(E[X])$." Only for affine $g$. Jensen gives the direction for convex and concave, and $E[X^2]>(E[X])^2$ whenever $X$ is non-degenerate.

"Chebyshev's bound is close to the truth." It is distribution-free and correspondingly loose. Its value is holding for every distribution with finite variance, not being tight for any.

"A density must be bounded." $\frac{1}{2\sqrt x}$ on $(0,1)$ is unbounded and integrates to $1$. Only the integral is constrained.

"$\operatorname{Var}(X+Y)=\operatorname{Var}X+\operatorname{Var}Y$." Needs uncorrelated, not independent — the weaker condition suffices, and options often demand the stronger.

The exam's angle

1. Check existence before computing. If the tail is like $1/x^2$, the mean may not exist and that is likely the question.
2. Get moments by differentiating the MGF at $0$, which is usually faster than integrating.
3. Use the characteristic function for sums and limits, since it always exists and multiplies over independent summands.
4. For a bound with no distributional assumption, reach for Chebyshev; for a comparison of $E[g(X)]$ and $g(E[X])$, check convexity and use Jensen.
5. Remember $\operatorname{Var}$ needs only uncorrelated to be additive.

The night before

• The CDF always exists and always determines the distribution.
• $E[X]$ exists iff $E|X|<\infty$. Cauchy has no mean, hence no variance and no MGF.
• $\phi(t)=E[e^{itX}]$ always exists; $M(t)=E[e^{tX}]$ need not.
• $M$ existing near $0$ determines the distribution and gives $M^{(n)}(0)=E[X^n]$.
• Moments alone do not determine a distribution — lognormal.
• Markov, Chebyshev ($1/k^2$, distribution-free and loose), Jensen ($E[g(X)]\ge g(E[X])$ for convex $g$), Cauchy–Schwarz.
• $\operatorname{Var}(X+Y)$ adds for uncorrelated variables.