Mathwords logoReference LibraryMathwords

Law of Large Numbers

The Law of Large Numbers is a theorem stating that as you repeat an experiment more and more times, the average of your results gets closer and closer to the expected (theoretical) value.

The Law of Large Numbers (LLN) states that as the number of independent, identically distributed trials nn increases, the sample mean xˉ\bar{x} converges to the population mean μ\mu. In its weak form, this convergence is in probability: for any positive number ε\varepsilon, P(∣xˉ−μ∣>ε)→0P(|\bar{x} - \mu| > \varepsilon) \to 0 as n→∞n \to \infty. The theorem requires that the random variable have a finite expected value.

Key Formula

P(∣Xˉn−μ∣>ε)→0as n→∞P\left(|\bar{X}_n - \mu| > \varepsilon\right) \to 0 \quad \text{as } n \to \infty
Where:
  • Xˉn\bar{X}_n = the sample mean after n trials
  • μ\mu = the population mean (expected value)
  • ε\varepsilon = any small positive number representing the allowed error
  • nn = the number of independent trials

Worked Example

Problem: You roll a fair six-sided die repeatedly. The theoretical mean is 3.5. Show how the sample mean approaches 3.5 as the number of rolls increases, given these cumulative results: after 5 rolls the sum is 22, after 50 rolls the sum is 165, and after 500 rolls the sum is 1743.
Step 1: Find the theoretical expected value of one die roll.
μ=1+2+3+4+5+66=3.5\mu = \frac{1+2+3+4+5+6}{6} = 3.5
Step 2: Calculate the sample mean after 5 rolls.
xˉ5=225=4.4\bar{x}_5 = \frac{22}{5} = 4.4
Step 3: Calculate the sample mean after 50 rolls.
xˉ50=16550=3.30\bar{x}_{50} = \frac{165}{50} = 3.30
Step 4: Calculate the sample mean after 500 rolls.
xˉ500=1743500=3.486\bar{x}_{500} = \frac{1743}{500} = 3.486
Step 5: Compare each sample mean to the theoretical mean of 3.5. The errors are ∣4.4−3.5∣=0.9|4.4 - 3.5| = 0.9, ∣3.30−3.5∣=0.20|3.30 - 3.5| = 0.20, and ∣3.486−3.5∣=0.014|3.486 - 3.5| = 0.014. As the number of trials grows, the sample mean gets much closer to 3.5.
Answer: The sample mean converges toward the theoretical mean of 3.5 as the number of rolls increases: 4.4 → 3.30 → 3.486, with the error shrinking from 0.9 to 0.014.

Visualization

Why It Matters

The Law of Large Numbers is the mathematical foundation behind how insurance companies set premiums, how casinos guarantee long-run profit, and how pollsters estimate public opinion from samples. In AP Statistics, it justifies using sample means to estimate population parameters — the larger your sample, the more reliable your estimate becomes.

Common Mistakes

Mistake: Believing the law applies to short runs — for example, thinking that after several heads in a row, tails is "due" to balance things out.
Correction: The LLN describes long-run averages, not individual outcomes. Each coin flip is independent. The average corrects itself through dilution by new data, not by future outcomes compensating for past ones. This error is called the Gambler's Fallacy.
Mistake: Confusing the Law of Large Numbers with the Central Limit Theorem.
Correction: The LLN says the sample mean approaches the population mean as nn grows. The Central Limit Theorem says the distribution of sample means becomes approximately normal as nn grows. They address different questions.

Related Terms

  • Probability — LLN links experimental probability to theoretical probability
  • Expected Value — The value the sample mean converges to
  • Bernoulli Trials — Independent repeated trials where LLN commonly applies