Likelihood ratio and standard tests

In one sentence

The likelihood ratio test compares the best fit under the null with the best fit overall, $-2\log\Lambda$ is asymptotically chi-square with degrees of freedom equal to the number of constraints, and every standard test is a special case.

Why the exam asks it

Because degrees of freedom are where the arithmetic goes wrong, and because Wilks's theorem has hypotheses — nested models, interior parameters — that options quietly drop.

The idea

The likelihood ratio test

$$\Lambda = \frac{\sup_{\theta\in\Theta_0} L(\theta)}{\sup_{\theta\in\Theta} L(\theta)} \in (0,1].$$

Small $\Lambda$ means the null fits much worse than the unrestricted model, so reject for small $\Lambda$.

Wilks: under $H_0$ and regularity, $-2\log\Lambda \xrightarrow{d} \chi^2_k$, where $k$ is the number of independent constraints the null imposes — equivalently the difference in the number of free parameters.

The hypotheses matter. The models must be nested, and the null must not lie on the boundary of the parameter space. Testing a variance component equal to zero violates the second, and the limit there is a mixture of chi-squares rather than a chi-square.

The standard tests as special cases

• One-sample $t$: $\dfrac{\bar X-\mu_0}{s/\sqrt n}\sim t_{n-1}$. One degree of freedom is spent estimating $\mu$.
• Two-sample $t$: $t_{n_1+n_2-2}$ with a pooled variance, assuming equal variances. Unequal variances give Welch's approximation.
• Paired $t$: a one-sample test on the differences, $t_{n-1}$ — using the two-sample test on paired data throws away the pairing and is a standard error.
• Chi-square goodness of fit: $\sum\frac{(O-E)^2}{E}\sim\chi^2_{k-1-p}$ for $k$ cells and $p$ estimated parameters. Each estimated parameter costs a degree of freedom.
• Contingency table: $\chi^2_{(r-1)(c-1)}$ — that count is the number of free cells once the margins are fixed.
• $F$ test for equality of variances, and for regression, as a ratio of independent chi-squares over their degrees of freedom.

The relationship worth knowing: $t_n^2 = F_{1,n}$. A two-sided $t$ test and the corresponding $F$ test are the same test, which is why regression output quotes one for a single coefficient and the other for a group.

p-values

The $p$-value is $P(\text{statistic at least this extreme}\mid H_0)$. Reject when $p<\alpha$.

It is not the probability the null is true, and it is not the probability the result was chance. Both misreadings invert the conditional, exactly as in the medical-test example.

A $p$-value is also a random variable: under $H_0$ it is uniform on $(0,1)$, which is why testing many hypotheses at level $\alpha$ produces false positives at rate $\alpha$ — the multiple comparisons problem, and the reason for Bonferroni and its relatives.

Where intuition breaks

"Degrees of freedom is the sample size." It is the number of free quantities after constraints. Each estimated parameter costs one, and the counting is what the question usually tests.

"$-2\log\Lambda$ is always chi-square." Asymptotically, for nested models with the null in the interior. Boundary cases give mixtures.

"A significant result means a large effect." Significance combines effect size with sample size. A tiny effect becomes significant with enough data.

"$p = 0.04$ and $p = 0.06$ are qualitatively different." They are nearly the same evidence. The threshold is a convention, not a boundary in nature.

"The chi-square goodness-of-fit test has $k-1$ degrees of freedom." $k-1-p$ where $p$ parameters were estimated from the data.

"$t$ and $F$ tests are different tools." $t_n^2 = F_{1,n}$. For a single parameter they are the same test.

"Non-significant means no effect." It means insufficient evidence at that sample size. Absence of evidence, as ever.

The exam's angle

1. Count degrees of freedom explicitly: cells or observations, minus constraints, minus estimated parameters.
2. For a likelihood ratio question, count the constraints — that is the chi-square df, and usually the whole question.
3. Check nesting and boundaries before applying Wilks.
4. Use $t^2=F$ to convert between the two when a question mixes them.
5. Read the $p$-value as a statement about data given the null, and reject any option that phrases it as a probability about the hypothesis.

The night before

• $\Lambda = \sup_{\Theta_0}L / \sup_\Theta L$; reject for small $\Lambda$.
• Wilks: $-2\log\Lambda\to\chi^2_k$, $k$ the number of constraints. Needs nested models and an interior null.
• One-sample $t_{n-1}$; two-sample $t_{n_1+n_2-2}$; paired is one-sample on differences.
• Goodness of fit $\chi^2_{k-1-p}$; contingency table $\chi^2_{(r-1)(c-1)}$.
• $t_n^2 = F_{1,n}$.
• $p$-value $=P(\text{data}\mid H_0)$; uniform under $H_0$; multiple testing inflates false positives.