Sufficiency, completeness, UMVUE, Cramér–Rao

In one sentence

A complete sufficient statistic plus unbiasedness gives the UMVUE outright, while the Cramér–Rao bound is a separate route that needs regularity conditions and fails exactly where the support depends on the parameter.

Why the exam asks it

Because $U(0,\theta)$ produces an unbiased estimator whose variance is below the Cramér–Rao expression, which looks like a contradiction and is not — the bound simply does not apply. That is the cleanest test of whether the hypotheses were learned along with the formula.

The idea

Sufficiency

$T$ is sufficient for $\theta$ if the conditional distribution of the sample given $T$ does not involve $\theta$ — all the information is in $T$.

Factorisation theorem: $T$ is sufficient iff $f(x;\theta) = g(T(x);\theta)\,h(x)$. That is how sufficiency is checked in practice — split the likelihood and see what $\theta$ touches.

Standard cases: $\sum X_i$ for Poisson, binomial and exponential; $(\sum X_i, \sum X_i^2)$ for normal with both parameters unknown; $X_{(n)}$ for $U(0,\theta)$.

Minimal sufficiency is the coarsest such $T$. Sufficient statistics are not unique — the whole sample is always sufficient and useless.

Completeness

$T$ is complete if $E_\theta[g(T)]=0$ for all $\theta$ forces $g\equiv0$ almost surely. It says $T$ carries no non-trivial estimator of zero, which is what makes unbiased functions of it unique.

Sufficiency does not imply completeness. The two are separate conditions and both are needed below.

The two theorems

Rao–Blackwell: for any unbiased $S$ and sufficient $T$, $E[S\mid T]$ is unbiased with variance no larger. Conditioning on a sufficient statistic never hurts and usually helps.

Lehmann–Scheffé: if $T$ is complete and sufficient and $g(T)$ is unbiased for $\theta$, then $g(T)$ is the unique UMVUE.

That is the constructive route: find a complete sufficient statistic, make an unbiased function of it, and you are done. No variance calculation is required.

The Cramér–Rao bound

For unbiased $T$ under regularity conditions,

$$\operatorname{Var}(T) \ge \frac{1}{nI(\theta)}, \qquad I(\theta) = E\left[\left(\frac{\partial}{\partial\theta}\log f\right)^2\right] = -E\left[\frac{\partial^2}{\partial\theta^2}\log f\right].$$

The crucial regularity condition is that the support does not depend on $\theta$, so that differentiation under the integral sign is valid.

Where it works: Poisson has $I(\theta)=1/\theta$, so the bound is $\theta/n$, and $\bar X$ has variance exactly $\theta/n$. Attained, so $\bar X$ is the UMVUE — that is the other route.

Where it does not: for $U(0,\theta)$ the support is $[0,\theta]$, which moves with the parameter. The UMVUE is $\frac{n+1}{n}X_{(n)}$ with variance $\dfrac{\theta^2}{n(n+2)}$. For $n=3$ that is $\theta^2/15$, while the Cramér–Rao expression would give $\theta^2/3$. The variance is five times smaller than the "bound", because there is no bound — the hypotheses fail.

Note the asymmetry: attaining the bound proves an estimator is UMVUE; failing to attain it proves nothing, since the bound may not be achievable by any estimator.

Where intuition breaks

"No unbiased estimator can beat the Cramér–Rao bound." $U(0,\theta)$ beats it comfortably, because the regularity conditions fail. Check the support before quoting the bound.

"Sufficient implies complete." Separate conditions. Lehmann–Scheffé needs both.

"The UMVUE is whatever has smallest variance." Among unbiased estimators. A biased estimator can have smaller mean squared error, and often does — shrinking towards zero trades bias for variance.

"An unbiased estimator always exists." Not always. And when it does, it need not be sensible: for Poisson, the only unbiased estimator of $e^{-2\theta}$ is $(-1)^X$, which takes the value $-1$ for a quantity that must be positive.

"Failing to attain the bound means the estimator is inefficient." It may mean the bound is unattainable. Efficiency is defined relative to what is achievable.

"The MLE is the UMVUE." Different criteria. The MLE need not be unbiased at all, as the next chapter shows.

"Fisher information is a property of the estimator." Of the model and the parameter. It measures how sharply the likelihood distinguishes nearby $\theta$.

The exam's angle

1. Factorise the likelihood to read off the sufficient statistic — this is one line and it starts every question here.
2. Check whether the support involves $\theta$. If it does, the Cramér–Rao bound is off the table and the answer is Lehmann–Scheffé.
3. For a UMVUE, find an unbiased function of the complete sufficient statistic, adjusting by a constant if needed — $\frac{n+1}{n}X_{(n)}$ is the model.
4. Compute $I(\theta)$ by the second-derivative form, which is usually easier than squaring the score.
5. Watch for "unbiased" in the option. Optimality claims here are always within that class.

The night before

• Factorisation: $f = g(T;\theta)h(x)$ characterises sufficiency.
• Complete: $E_\theta[g(T)]=0\ \forall\theta \Rightarrow g\equiv0$. Not implied by sufficiency.
• Rao–Blackwell: conditioning on a sufficient statistic cannot increase variance.
• Lehmann–Scheffé: unbiased function of a complete sufficient statistic is the unique UMVUE.
• Cramér–Rao: $\operatorname{Var}\ge1/(nI(\theta))$, requiring the support to be free of $\theta$.
• $U(0,\theta)$: UMVUE $\frac{n+1}{n}X_{(n)}$, variance $\theta^2/(n(n+2))$ — below the nominal bound, because the bound does not apply.
• Poisson: $I=1/\theta$, bound $\theta/n$, attained by $\bar X$.
• Attaining the bound proves UMVUE; not attaining it proves nothing.