SRS, stratified and systematic sampling

In one sentence

Simple random sampling is the baseline, stratification beats it when the strata differ, and systematic sampling is the only one of the three that can be worse — catastrophically so if the list has a period.

Why the exam asks it

Because the variance formulas are short and their differences are the whole content, and because the finite population correction is a factor people drop.

The idea

Simple random sampling

Without replacement, from a population of $N$ with variance $S^2$:

$$\operatorname{Var}(\bar y) = \frac{S^2}{n}\cdot\frac{N-n}{N}.$$

The last factor is the finite population correction. It equals $1$ when $n\ll N$, and $0$ when $n=N$ — a census has no sampling error, which is the sanity check that the formula is right.

With replacement there is no correction: $\operatorname{Var}=\sigma^2/n$. Sampling without replacement is therefore more precise, since you never waste a draw on a unit already seen.

The sample mean is unbiased for the population mean under both.

Stratified sampling

Divide into $L$ strata of sizes $N_h$, sample $n_h$ from each. With $W_h=N_h/N$:

$$\bar y_{st}=\sum_h W_h\bar y_h, \qquad \operatorname{Var}(\bar y_{st})=\sum_h W_h^2\frac{S_h^2}{n_h}\cdot\frac{N_h-n_h}{N_h}.$$

Only within-stratum variance appears. Stratification removes the between-strata component entirely, which is why it pays exactly when the strata means differ — and why choosing strata that are internally homogeneous is the whole design skill.

Allocations:

• Proportional: $n_h\propto N_h$. Simple, and never worse than SRS.
• Neyman: $n_h\propto N_hS_h$ — more sample where the variance is larger. Minimises the variance for fixed total $n$.
• Optimal with costs: $n_h\propto N_hS_h/\sqrt{c_h}$.

Ordering: $\operatorname{Var}_{\text{Neyman}} \le \operatorname{Var}_{\text{proportional}} \le \operatorname{Var}_{\text{SRS}}$, with equality when the strata are identical.

Systematic sampling

Take every $k$-th unit after a random start, with $k=N/n$. Easy to execute, and equivalent to choosing one cluster at random from $k$ possible ones.

• If the list is in random order, it behaves like SRS.
• If the list is ordered by the variable of interest, it is better than SRS — the sample spreads across the range, like implicit stratification.
• If the list has a period matching $k$, it can be far worse. Sampling every $7$th day gives the same weekday every time; sampling every $10$th house on a street of ten-house blocks gives the corner house every time.

Systematic sampling is the only one of the three that can beat or lose to SRS, and the periodic case is the standard question.

A practical difficulty: with a single random start there is no unbiased variance estimator from one sample, which is why multiple random starts are used when precision must be reported.

Where intuition breaks

"Sampling with replacement is more precise." Without replacement is more precise — the finite population correction is below $1$.

"The correction matters only for small populations." It matters when $n/N$ is not small. Sampling a quarter of a population cuts the variance by $3/4$.

"Stratification always helps." It helps when strata means differ. With identical strata it matches SRS and never beats it.

"Neyman allocation is proportional to stratum size." Proportional to $N_hS_h$ — size times standard deviation. A small but variable stratum earns extra sample.

"Systematic sampling is always at least as good as SRS." The periodic case can be much worse, and it is the only design here that can lose.

"Systematic sampling is random sampling." One random start selects the whole sample, so it is effectively a cluster sample of size one.

"A larger sample always reduces variance proportionally." With the correction, the gain accelerates as $n$ approaches $N$ and reaches zero variance at a census.

The exam's angle

1. Write the variance with the correction and check the $n=N$ case gives zero.
2. For stratified questions, note that only within-stratum variance appears — that single fact explains when stratification pays.
3. Distinguish the allocations: proportional uses $N_h$, Neyman uses $N_hS_h$.
4. For systematic sampling, ask whether the list has structure. Ordered helps, periodic hurts.
5. Compare designs by variance, not by intuition — the ordering Neyman $\le$ proportional $\le$ SRS is what the options test.

The night before

• SRS without replacement: $\operatorname{Var}(\bar y)=\frac{S^2}{n}\cdot\frac{N-n}{N}$; the fpc is $1$ for $n\ll N$ and $0$ at a census.
• Without replacement is more precise than with.
• Stratified variance involves only within-stratum variance; stratification pays when strata means differ.
• Proportional $n_h\propto N_h$; Neyman $n_h\propto N_hS_h$; with costs, divide by $\sqrt{c_h}$.
• Neyman $\le$ proportional $\le$ SRS.
• Systematic: like SRS when the list is random, better when ordered, much worse when periodic. One random start, so effectively a single cluster.