Sufficiency, completeness, UMVUE, Cramér–Rao

Sufficiency

Factorisation theorem: T is sufficient ⇔ f(x|$\theta) = g(T(x), \theta)h(x)$.

Family	Sufficient statistic
$N(\mu, \sigma^{2})$, both unknown	$(\sum{}X_{i}, \sum{}X_{i}^{2})$
Bernoulli/Binomial/Poisson	$\sum{}X_{i}$
Uniform$(0, \theta)$	$X_{(n)}$
Uniform$(\theta, \theta+1)$	$(X_{(1)}, X_{(n)})$
Shifted exponential $e^{\theta-x}$, $x \ge \theta$	$X_{(1)}$
Exponential family	the natural statistic

Minimal sufficient: use the ratio criterion — f(x|$\theta)/f(y$|$\theta)$ is $\theta-$free ⇔ T(x) = T(y).

Completeness

T is complete if $E_\theta[g(T)] = 0$ for all $\theta \Rightarrow g \equiv 0$. Full-rank exponential families and the order statistics of Uniform$(0,\theta)$ are complete. **Uniform$(-\theta, \theta)$ is not$**: X_{(n)}$ is sufficient but $\sum{}X_{i}$ has zero expectation without being zero.

The UMVUE machine

• Rao–Blackwell: conditioning any unbiased estimator on a sufficient T improves it.
• Lehmann–Scheffé: if T is complete sufficient and g(T) is unbiased, then g(T) is the unique UMVUE.

Standard results:

Model	Parameter	UMVUE
Uniform$(0,\theta)$	$\theta$	$(n+1)X_{(n)}/n$
Shifted exp $e^{\theta-x}$	$\theta$	$X_{(1)} - 1/n$
$N(\mu,\sigma^{2})$	$\mu$	X̄
$\theta{}x^{\theta-1}$ on (0,1)	$1/\theta$	$-(1/n)\sum{}\ln X_{i}$
Bernoulli(p)	p	X̄

Cramér–Rao

$Var(T) \ge [\psi'(\theta)]^{2}/($nI$(\theta))$ for unbiased T of $\psi(\theta)$, under regularity. Equality ⇔ the model is exponential family with T the natural statistic.

Regularity fails when the support depends on $\theta ($uniform, shifted exponential) — there the UMVUE can beat the "bound", which is why those examples appear so often.

Key takeaways

• Factorisation ⇒ sufficient; ratio criterion ⇒ minimal.
• Complete + sufficient + unbiased = UMVUE (Lehmann–Scheffé).
• Cramér–Rao does not apply when the support moves with $\theta$.