Multivariate normal distribution

In one sentence

The multivariate normal is closed under every linear operation, its conditionals are normal with a linear mean, and it is the one family where zero correlation really does mean independence.

Why the exam asks it

Because "normal marginals" and "jointly normal" are different conditions and the difference decides the independence question, and because the conditional-distribution formula is quotable and therefore examinable.

The idea

$\mathbf{X}\sim N_p(\boldsymbol\mu,\Sigma)$ has density

$$f(\mathbf{x}) = \frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}}\exp\left(-\tfrac12(\mathbf{x}-\boldsymbol\mu)'\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu)\right)$$

when $\Sigma$ is positive definite. If $\Sigma$ is singular the distribution is degenerate — it lives on a lower-dimensional subspace and has no density, which happens exactly when a linear combination of the components is constant.

The cleanest definition avoids the density: $\mathbf{X}$ is multivariate normal iff $\mathbf{a}'\mathbf{X}$ is univariate normal for every vector $\mathbf{a}$. That formulation covers the degenerate case and makes the closure properties obvious.

The properties

• Linear transformations: $A\mathbf{X}+\mathbf{b}\sim N(A\boldsymbol\mu+\mathbf{b}, A\Sigma A')$. Any linear map of a normal vector is normal.
• Marginals are normal — take $A$ to select coordinates.
• Conditionals are normal: $$\mathbf{X}_1\mid\mathbf{X}_2=\mathbf{x}_2 \sim N\left(\boldsymbol\mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(\mathbf{x}_2-\boldsymbol\mu_2),\ \Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}\right).$$ The mean is linear in the conditioning value and the variance does not depend on it — which is exactly the linear-regression model, and the reason regression and the multivariate normal are the same subject.
• Uncorrelated implies independent: $\Sigma$ block-diagonal makes the density factor. This is the exception to the general rule from the joint-distributions chapter.
• Quadratic form: $(\mathbf{X}-\boldsymbol\mu)'\Sigma^{-1}(\mathbf{X}-\boldsymbol\mu)\sim\chi^2_p$ — the squared Mahalanobis distance, and the basis of confidence ellipsoids.

In the bivariate case with correlation $\rho$, the conditional variance is $\sigma_1^2(1-\rho^2)$: conditioning removes the fraction $\rho^2$ of the variance, which is exactly $R^2$ in the one-predictor regression.

The trap

Normal marginals do not imply joint normality. The standard construction: take $(X,Y)$ standard bivariate normal with $\rho=0$, then flip the sign of $Y$ whenever $XY<0$. Both marginals remain standard normal, the pair is uncorrelated, and they are strongly dependent — $Y$'s sign is determined by $X$'s.

So "uncorrelated normals are independent" is false as usually stated. The correct statement needs joint normality, and options omit the word.

Where intuition breaks

"Normal marginals means jointly normal." False, and it is the hypothesis the independence result depends on.

"Uncorrelated normals are independent." Only if jointly normal.

"Every multivariate normal has a density." Singular $\Sigma$ gives a degenerate distribution on a subspace, with no density.

"The conditional variance depends on the conditioning value." It does not — that is the homoscedasticity assumption of linear regression, and here it is a theorem rather than an assumption.

"$\Sigma$ can be any symmetric matrix." It must be positive semi-definite. A symmetric matrix with a negative eigenvalue is not a covariance matrix.

"A linear combination of normals is normal." Of jointly normal components. Two separately normal variables can have a non-normal sum.

"The Mahalanobis distance is chi-square with $p-1$ degrees of freedom." $p$, when $\mu$ and $\Sigma$ are known. Estimating them changes the distribution — that is Hotelling's $T^2$.

The exam's angle

1. Use the linear-combination definition — it settles closure questions immediately.
2. For conditionals, quote the formula and note the variance is free of the conditioning value.
3. Check $|\Sigma|$ — zero means degenerate, and questions use that to test whether a density exists.
4. When independence is claimed from zero correlation, look for "jointly".
5. For confidence regions, use the chi-square quadratic form with $p$ degrees of freedom.

The night before

• $\mathbf{X}$ is multivariate normal iff every $\mathbf{a}'\mathbf{X}$ is univariate normal.
• $A\mathbf{X}+\mathbf{b}\sim N(A\mu+\mathbf{b}, A\Sigma A')$; marginals normal.
• Conditional: mean $\mu_1+\Sigma_{12}\Sigma_{22}^{-1}(\mathbf{x}_2-\mu_2)$, variance $\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$ — free of $\mathbf{x}_2$.
• Bivariate: conditional variance $\sigma_1^2(1-\rho^2)$.
• Uncorrelated $\Rightarrow$ independent, for jointly normal only. Normal marginals are not enough.
• Singular $\Sigma$ $\Rightarrow$ degenerate, no density.
• $(\mathbf{X}-\mu)'\Sigma^{-1}(\mathbf{X}-\mu)\sim\chi^2_p$.