The Timpson Lyceum · Discrete Math (2025)

Probability, Expectation, and Variance - Lecture 17 (Discrete Math)

Probability, Expectation, and Variance – Lecture 17

Overview

This lecture builds the theory of real-valued random variables on top of the probability model from earlier in the course: a sample space $S$ of outcomes together with a probability function $P$. It defines the expected value of a random variable, proves the linearity properties that make expectations easy to compute, and shows why the product rule $E[XY] = E[X]\,E[Y]$ holds only for independent variables. It then introduces variance as a measure of how spread out a distribution is and derives the shortcut formula that everyone uses in practice. Two recurring intuitions tie it together: expectation as a long-run average (and as a center of mass), and variance as average squared distance from that center.

Key Concepts

Sample space and probability function

• A sample space is a pair $(S, P)$ where $S$ is the set of all possible outcomes and $P$ is the probability function.
• Restrictions on $P$: probabilities are never negative, and they sum to $1$, i.e. $\sum_{s \in S} P(s) = 1$.

Real random variable

• A real random variable $X$ is a function whose domain is the sample space $S$ and whose codomain is $\mathbb{R}$: it maps each outcome to a real number.
• Example (one fair die): $S = \{1,2,3,4,5,6\}$, $P(s) = \tfrac{1}{6}$ for every outcome, and $X(s) = s$ returns the number rolled.

Expected value

• Definition: $E[X] = \sum_{s \in S} X(s)\,P(s)$ — for every outcome, multiply its value by its probability and add up the results.
• Interpretation: the long-run average value of $X$, or the "fair price" of a game that pays $X(s)$ dollars for outcome $s$. A single fair die has $E[X] = 3.5$.
• The shorthand $\mu$ denotes $E[X]$ (which variable is understood from context).

Second formula for expectation (grouping by output)

• Instead of summing over outcomes, group by the values $X$ can produce: $E[X] = \sum_{a} a\,P(X = a).$
• For two dice with $X =$ sum, the outputs run $2$ through $12$ with probabilities $\tfrac{1}{36}, \tfrac{2}{36}, \dots, \tfrac{6}{36}, \dots, \tfrac{1}{36}$, giving $E[X] = \tfrac{252}{36} = 7$.

Linearity of expectation

• Sum rule: $E[X + Y] = E[X] + E[Y]$ — proved by distributing $P(s)$ and splitting the sum.
• Scalar rule: $E[cX] = c\,E[X]$ for a constant $c \in \mathbb{R}$ (factor the constant out of the sum).
• Combined: $E[aX + bY] = a\,E[X] + b\,E[Y]$.
• Payoff: the expected sum of two dice is $3.5 + 3.5 = 7$ without summing all $36$ outcomes; for $10$ dice it is $10 \times 3.5 = 35$.

Product rule (independence required)

• $E[XY] = E[X]\,E[Y]$ holds only when $X$ and $Y$ are independent, meaning $P(X = b \text{ and } Y = c) = P(X = b)\,P(Y = c)$.
• Counterexample (flip a coin twice): with $X_H =$ number of heads and $X_T =$ number of tails, $E[X_H] = E[X_T] = 1$ so $E[X_H]\,E[X_T] = 1$, but $E[X_H X_T] = \tfrac{1}{2}$. The variables are dependent, so the rule fails.

Center of mass

• Expectation also computes a center of mass: place masses (here, equal "weights" acting like probabilities) at positions, and $E[X]$ is the balance point.

Variance

• Variance measures how spread out a distribution is — average squared distance from the center $\mu$.
• Definition: $\operatorname{Var}(X) = E\big[(X - \mu)^2\big] = \sum_{s \in S} \big(X(s) - \mu\big)^2 P(s)$.
• The deviation is squared (not absolute-valued) to keep terms positive without a piecewise function, to magnify large deviations, and to shrink small ones. The standard deviation is $\sqrt{\operatorname{Var}(X)}$.
• Three mean-zero variables illustrate the idea: $X \in \{-2, 2\}$ each with prob. $\tfrac{1}{2}$ has $\operatorname{Var} = 4$; a variable taking $\pm 10$ with prob. $0.001$ each (and $0$ otherwise) has $\operatorname{Var} = 0.2$; a variable taking $-5, 0, 5$ each with prob. $\tfrac{1}{3}$ has $\operatorname{Var} = \tfrac{50}{3} \approx 16.7$.

Computational formula for variance

• $\operatorname{Var}(X) = E[X^2] - \big(E[X]\big)^2$. Note $E[X^2]$ (square the values before averaging) differs from $\mu^2 = (E[X])^2$.
• This is far easier in practice: you already need $E[X]$, and $E[X^2]$ is a direct expectation rather than a sum of squared deviations.

Worked Example

Compute the variance of a single fair die roll, $S = \{1,\dots,6\}$, $P(s) = \tfrac{1}{6}$, $X(s) = s$.

Use $\operatorname{Var}(X) = E[X^2] - (E[X])^2$. First find $E[X^2] = \sum_s X(s)^2 P(s)$, then subtract the square of the known mean $E[X] = \tfrac{7}{2}$:

Summary

• A real random variable maps outcomes in $S$ to real numbers; its expected value is $E[X] = \sum_{s} X(s)P(s)$, the long-run average.
• Expectation is linear: $E[aX + bY] = a\,E[X] + b\,E[Y]$, which collapses huge sums (e.g. $10$ dice $\to 35$) into trivial arithmetic.
• The product rule $E[XY] = E[X]E[Y]$ requires independence; the two-coin-flip example shows it fails for dependent variables.
• Expectation doubles as a center of mass (and a weighted average) of masses placed along a line.
• Variance $\operatorname{Var}(X) = E[(X-\mu)^2]$ measures spread; squaring (rather than absolute value) avoids piecewise definitions and exaggerates outliers.
• In practice always compute variance with $\operatorname{Var}(X) = E[X^2] - (E[X])^2$ — e.g. a single fair die has variance $\tfrac{35}{12} \approx 2.92$.