Probability

1. Axioms, conditional probability, independence, Bayes

Exam focus: Most Part-C items here are symmetry checks: P(A|B) > P(A) ⇔ P(A∩B) > P(A)P(B) is symmetric in A and B, while P(A|B) > P(B) is not.

Lec-03 Introduction to Probability

NPTEL · Probability and Statistics

Lec-04 Laws of Probability - I

NPTEL · Probability and Statistics

Conditional probability, independence and Bayes.

Probability axioms, conditioning and independence free

Basics

P(A ∪ B) = P(A) + P(B) − P(A∩B); inclusion–exclusion for more sets. **Boole $* * : P (\cup A_{i}) \leq \sum P (A_{i}) . * *$ Bonferroni $* * : P (\cap A_{i}) \geq 1 - \sum P (A_{i}$ ᶜ).

Conditioning

P(A|B) = P(A∩B)/P(B). **Bayes $* * : P (A_{i}$ |B) = P(B| $A_{i}) P (A_{i}) / \sum P (B$ | $A_{j}) P (A_{j})$ .

Symmetry facts (the recurring Part-C pattern):

Statement	Symmetric in A, B?
P(A\|B) = 0 ⇔ P(B\|A) = 0	✓ (both say P(A∩B) = 0)
P(A\|B) > P(A) ⇔ P(B\|A) > P(B)	✓ (both say P(A∩B) > P(A)P(B))
P(A\|B) = 1 ⇒ P(B\|A) = 1	✗ (A ⊇ B does not give B ⊇ A)
P(A\|B) > P(B) ⇒ P(B\|A) > P(A)	✗

Independence

A ⫫ B ⇔ P(A∩B) = P(A)P(B). Then Aᶜ ⫫ B too.
Pairwise independence does not imply mutual independence: two fair coins with A = first head, B = second head, C = "same face" are pairwise independent but not mutually.
Independent events with positive probability are never disjoint.

Standard traps

Mutually exclusive ≠ independent (they are opposites unless one has probability 0).
Conditional independence given C does not imply independence, and vice versa (Simpson's paradox territory).

Key takeaways

Convert every conditional statement to P(A∩B) vs P(A)P(B) and symmetry becomes obvious.
Pairwise ⇏ mutual independence.
Disjoint and independent are almost opposite notions.

Part C2023 Juneshow ▾

Let A, B be two events in a discrete probability space with P(A) > 0 and P(B) > 0. Which of the following are necessarily true?

A.If P(A | B) = 0 then P(B | A) = 0.✓
B.If P(A | B) = 1 then P(B | A) = 1.
C.If P(A | B) > P(A) then P(B | A) > P(B).✓
D.If P(A | B) > P(B) then P(B | A) > P(A).

Solution

(1) P(A∩B) = 0 is symmetric. (3) P(A|B) > P(A) ⇔ P(A∩B) > P(A)P(B), symmetric in A, B. (2) A ⊇ B (a.s.) does not give B ⊇ A. (4) Take B ⊂ A with P(B) small.

Save for spaced revision (free account)

Part Cshow ▾

A test for a disease has sensitivity 0.99 and specificity 0.95, and the disease affects 1% of the population. Which of the following are true?

A.P(test positive) = 0.0594.✓
B.P(diseased | positive) = 1/6.✓
C.P(diseased | positive) > 1/2.
D.Halving the prevalence to 0.5% lowers P(diseased | positive) further.✓

Solution

P(+) = 0.99×0.01 + 0.05×0.99 = 0.0099 + 0.0495 = 0.0594. Bayes: P(D|+) = 0.0099/0.0594 = 1/6 ≈ 0.17, nowhere near 1/2 — the base-rate fallacy in numbers. Lower prevalence shrinks the true-positive share of the positives, dropping the posterior further.

Save for spaced revision (free account)

Part Cshow ▾

Roll a fair die once; let A = {even}, B = {1, 2}, C = {1, 2, 3, 4}. Which of the following are true?

A.A and B are independent.✓
B.A and C are independent.✓
C.P(A | B) = 1/2.✓
D.Pairwise independence of three events implies their mutual independence.

Solution

P(A) = 1/2, P(B) = 1/3, P(A∩B) = P({2}) = 1/6 = P(A)P(B). P(C) = 2/3, P(A∩C) = P({2, 4}) = 1/3 = P(A)P(C). And P(A|B) = (1/6)/(1/3) = 1/2. (4) is false — the standard counterexample is two fair coins with A = first heads, B = second heads, C = same outcome: pairwise independent, but P(A∩B∩C) = 1/4 ≠ 1/8.

Save for spaced revision (free account)

Part Bshow ▾

Let P(A) = 0.6 and P(B) = 0.5. The smallest possible value of P(A ∩ B) is

A.0
B.0.1✓
C.0.3
D.0.5

Solution

P(A ∩ B) = P(A) + P(B) − P(A ∪ B) ≥ 0.6 + 0.5 − 1 = 0.1, since P(A ∪ B) ≤ 1. The bound is attained when A ∪ B fills the whole space. (The maximum, by contrast, is min(0.6, 0.5) = 0.5.)

Save for spaced revision (free account)

2. Random variables, distributions, moments, MGF

Exam focus: MGF determines the distribution and factorises over independent sums. Know which distributions have no MGF (Cauchy, t) and which moments fail to exist.

Lec-07 Random Variables

NPTEL · Probability and Statistics

Lec-08 Probability Distributions

NPTEL · Probability and Statistics

Lec-09 Characteristics of Distributions

NPTEL · Probability and Statistics

Moments and generating functions.

3. Standard discrete and continuous distributions

Exam focus: Know the mean/variance table cold and the standard relationships (sum of exponentials = gamma, min of exponentials = exponential, memorylessness).

Lec-10 Special Distributions - I

NPTEL · Probability and Statistics

The standard families; keep the mean/variance table beside you.

Lec-11 Special Distributions - II

NPTEL · Probability and Statistics

The standard distributions free

Reference table

Distribution	Mean	Variance
Bernoulli(p)	p	pq
Binomial(n,p)	np	npq
Poisson $(λ)$	$λ$	$λ$
Geometric (trials, p)	1/p	$q / p^{2}$
Negative binomial(r,p)	r/p	rq $/ p^{2}$
Hypergeometric	nK/N	nK $(N - K) (N - n) / (N^{2} (N - 1))$
Uniform(a,b)	(a+b)/2	$(b - a)^{2} /12$
Exponential $(λ)$	$1/ λ$	$1/ λ^{2}$
Gamma $(k, λ)$	$k / λ$	$k / λ^{2}$
Beta(a,b)	a/(a+b)	ab $/ ((a + b)^{2} (a + b + 1))$
$N (μ, σ^{2})$	$μ$	$σ^{2}$
$χ_{n}^{2}$	n	2n
$t_{ν}$	$0 (ν > 1)$	$ν / (ν - 2) (ν > 2)$
Cauchy	—	—

Relationships to quote

Sum of n i.i.d. Exp $(λ) =$ Gamma $(n, λ)$ ; Gamma(n/2, ½ $) = χ_{n}^{2}$ .
**min of independent Exp $(λ_{i})$ ~ Exp $(\sum λ_{i}) * *$ — the series-system fact.
Sum of independent Poissons is Poisson; of independent normals is normal; of independent $χ^{2}$ is $χ^{2}$ .
Z ~ $N (0, 1) \Rightarrow Z^{2}$ ~ $χ_{1}^{2} . t = Z / χ_{n}^{2} / n; F = (χ_{m}^{2} / m) / (χ_{n}^{2} / n)$ .
Binomial → Poisson (n large, np fixed) → Normal (CLT).
Memorylessness characterises the exponential (continuous) and geometric (discrete): P(X > s+t | X > s) = P(X > t).

Order statistics

For i.i.d. with CDF $F : P (X_{(n)} \leq x) = F (x)^{n}$ and $P (X_{(1)} > x) = (1 - F (x))^{n}$ . For Uniform $(0, 1), X_{(k)}$ ~ Beta(k, n−k+1).

Key takeaways

Exponential: memoryless, minimum stays exponential, sum becomes gamma. $- χ^{2}, t$ and F are all built from normals — know the constructions.
Uniform order statistics are Beta.

Part C2023 Juneshow ▾

Let

X_{1}

and

X_{2}

be independent,

X_{1}

gamma with mean 10 and variance 10, and

X_{2}

~ N(3, 4). Let

f_{1}, f_{2}

be their densities. Define Y with density

f (y) = 0.4 f_{1} (y) + q f_{2} (y)

. Which of the following are true?

A.q = 0.6✓
B.E[Y] = 5.8✓
C.Var(Y) = 3.04
D. $Y = 0.4 X_{1} +$ qX $_{2}$

Solution

Densities integrate to 1 ⇒ q = 0.6. Mixture mean $0.4 \cdot 10 + 0.6 \cdot 3 = 5.8. E [Y^{2}] = 0.4 (10 + 100) + 0.6 (4 + 9) = 51.8$ , so $V a r (Y) = 51.8 - 5. 8^{2} = 18.16. A$ mixture is not a linear combination of the variables.

Save for spaced revision (free account)

Part B2023 Juneshow ▾

Let

X_{1}, X_{2}, X_{3}, X_{4}

be i.i.d. Bernoulli(1/3) and

X_{(1)} \leq \dots \leq X_{(4)}

the order statistics. Which of the following is true?

A. $X_{(1)}$ and $X_{(4)}$ are independent.
B.Expectation of $X_{(2)}$ is 1/2.
C.Variance of $X_{(2)}$ is 8/81.✓
D. $X_{(4)}$ is a degenerate random variable.

Solution

$X_{(2)} = 1$ iff at least three of the four are $1 : P = 4 (1/3)^{3} (2/3) + (1/3)^{4} = 9/81 = 1/9. A$ Bernoulli(1/9) has variance (1/9)(8/9) = 8/81.

Save for spaced revision (free account)

Part Cshow ▾

Let X follow a Poisson distribution with parameter

λ > 0

. Which of the following are true?

A. $E [X] = V a r (X) = λ$ .✓
B.The moment generating function of X is $exp (λ (e$ ᵗ − 1)).✓
C.X takes negative values with positive probability.
D.The distribution of X is symmetric about its mean.

Solution

(1) Both the mean and the variance of a Poisson variable equal $λ$ — the equality of the two is the distribution's signature, and a sample variance far from the sample mean is the usual evidence against a Poisson model. $(2) M (t) = \sum$ $e^{t k}$ $e^{- λ} λ^{k} / k! =$ $e^{- λ} \sum (λ e$ ᵗ $)^{k} / k! = exp (λ (e$ ᵗ − 1)). (3) The support is {0, 1, 2, …}, so negative values have probability 0. (4) The distribution is right-skewed, with skewness $1/\sqrt λ > 0$ ; it only becomes approximately symmetric as $λ \to \infty$ .

Save for spaced revision (free account)

Part Cshow ▾

Let X ~ Exponential

(λ)

, with density

λ e^{- λ x}

on x > 0. Which of the following are true?

A.P(X > s + t | X > s) = P(X > t) for all s, t > 0.✓
B. $E [X] = 1/ λ$ and $V a r (X) = 1/ λ^{2}$ .✓
C.For independent exponentials with rates $λ_{1}$ and $λ_{2}, min (X_{1}, X_{2})$ is exponential with rate $λ_{1} + λ_{2}$ .✓
D.The exponential is the only continuous distribution on $(0, \infty)$ with the property in option 1.✓

Solution

(1) Memorylessness: e^{ $- λ (s + t)$ }/e^{ $- λ s$ } = $e^{- λ t}$ . (2) Standard moments. (3) P(min > t) = $e^{- λ_{1} t}$ · $e^{- λ_{2} t}$ . (4) The survival function must satisfy G(s + t) = G(s)G(t), and the only monotone solutions are exponentials — the characterisation theorem. All four hold.

Save for spaced revision (free account)

Part Bshow ▾

A fair coin is tossed repeatedly. The expected number of tosses needed to obtain the first head is

A.1
B.2✓
C.3
D.4

Solution

The number of tosses is geometric with p = 1/2, whose mean is 1/p = 2. First-step reasoning gives the same: E = 1 + (1/2)·0 + (1/2)·E, so E = 2.

Save for spaced revision (free account)

Part Bshow ▾

Let X ~

N (μ, σ^{2})

. The probability P(|

X - μ

< σ)

is approximately

A.0.50
B.0.68✓
C.0.90
D.0.95

Solution

Standardising, P(|Z| < 1) ≈ 0.6827 — the '68' of the 68–95–99.7 rule. Two standard deviations would give ≈ 0.95.

Save for spaced revision (free account)

4. Joint distributions, transformations, order statistics

Exam focus: Uncorrelated is weaker than independent except for the joint normal. Use the Jacobian formula for transformations and the standard order-statistic densities.

Lec-18 Joint Distributions - I

NPTEL · Probability and Statistics

Lec-22 Transformations of Random Vectors

NPTEL · Probability and Statistics

The Jacobian method for transformations.

Probability

1. Axioms, conditional probability, independence, Bayes

Bayes and the base rate: why a 99% accurate test is usually wronginteractivefree

Probability axioms, conditioning and independence free

Basics

Conditioning

Independence

Standard traps

Key takeaways

2. Random variables, distributions, moments, MGF

Moments, MGFs, and when they fail to existinteractive

Random variables, moments and generating functions

Moments

MGF and characteristic function

3. Standard discrete and continuous distributions

The signatures that identify a distribution instantlyinteractive

The standard distributions free

Reference table

Relationships to quote

Order statistics

Key takeaways

4. Joint distributions, transformations, order statistics

Order statistics from first principlesinteractive

Joint distributions, transformations and order statistics

Independence vs correlation

Transformations