Bayesian Inference — Definition, Formula & Example
Bayesian inference is a way of doing statistics where you start with a belief about an unknown quantity, collect data, and then update that belief. The result is a probability for each possible value of the unknown, adjusted by the evidence.
Bayesian inference treats an unknown parameter as a random variable with a prior distribution . After observing data , it computes the posterior distribution by Bayes' theorem: the posterior is proportional to the likelihood times the prior. All estimates, such as posterior means and credible intervals, come from this posterior.
Key Formula
Where:
- = The unknown parameter or hypothesis
- = The observed data
- = Prior: your probability for before seeing the data
- = Likelihood: how probable the data are if is true
- = Posterior: your updated probability for after seeing the data
- = Evidence: total probability of the data, which makes the posteriors sum to 1. For a continuous parameter, the sum becomes an integral.
Worked Example
Problem: A factory machine is either in good condition, with a 10% defect rate, or worn, with a 40% defect rate. Maintenance records give a prior of and . You inspect 2 parts, chosen independently, and both are defective. What is the posterior probability that the machine is worn?
Step 1: Find the likelihood of two defects under each hypothesis. Square each defect rate because the parts are independent.
Step 2: Multiply each likelihood by its prior. Then add the products to get the evidence .
Step 3: Divide each product by the evidence to get the posterior probabilities.
Answer: The posterior probability that the machine is worn is 0.8. Two defects moved your belief from 20% to 80%. The prior and the posterior have swapped.
Why It Matters
Bayesian inference gives a direct probability that a hypothesis is true, which a p-value cannot do. It powers spam filters, A/B testing at tech companies, and clinical trials that update as patients enroll. It is also the core of courses in Bayesian data analysis and machine learning.
Common Mistakes
Mistake: Treating the likelihood as if it were the posterior .
Correction: In the example, two defects have probability 0.16 if the machine is worn. The probability the machine is worn given two defects is 0.8. These are different quantities. To reverse the conditioning, you must weight by the prior and divide by .
