Response Bias — Definition, Formula & Examples
Response bias is a tendency for survey respondents to give inaccurate or untruthful answers, causing the collected data to systematically differ from the true values. It arises from the way questions are worded, who asks them, or social pressure on the respondent.
Response bias is a form of systematic error in survey data that occurs when the measurement process itself — including question wording, question order, interviewer characteristics, or the respondent's desire to appear socially acceptable — causes responses to deviate from respondents' true beliefs, behaviors, or attributes in a consistent direction.
How It Works
Response bias distorts results even when every person in the sample actually responds. The bias enters through the measurement instrument rather than through who is or is not included in the sample. Common sources include leading questions ("Don't you agree that…?"), sensitive topics where people underreport undesirable behavior (like drug use) or overreport desirable behavior (like voting), and interviewer influence where respondents tailor answers to please the person asking. To reduce response bias, researchers use neutral wording, anonymous surveys, randomized response techniques, and careful pilot testing of questionnaires.
Worked Example
Problem: A school administrator surveys 200 students face-to-face and asks: "How many hours did you study last week?" The sample mean is 14 hours. An anonymous follow-up survey of the same 200 students yields a mean of 9 hours. Identify the direction and approximate size of the response bias.
Identify the source of bias: Students were asked in person about a socially desirable behavior (studying). They likely overstated their study time to appear more diligent.
Calculate the bias: Response bias equals the difference between the observed (biased) mean and the more accurate (anonymous) mean.
State the direction: The face-to-face survey systematically overestimated study time by about 5 hours per student. This is an upward (positive) response bias driven by social desirability.
Answer: The survey overestimated average study time by approximately 5 hours due to response bias from social desirability in a face-to-face setting.
Another Example
Problem: A political poll asks: "Given the mayor's failed economic policies, do you approve of her performance?" Of 500 respondents, 28% approve. Explain why this result may exhibit response bias.
Identify the leading language: The phrase "failed economic policies" is loaded — it frames the mayor negatively before asking for an opinion.
Predict the effect: The wording pushes respondents toward disapproval, so the 28% approval rate is likely lower than what a neutrally worded question would produce.
Suggest a fix: A neutral version might read: "Do you approve or disapprove of the mayor's job performance?" This lets respondents form their own judgment without influence from the question's framing.
Answer: The leading question biases responses downward. The true approval rate is likely higher than 28%.
Visualization
Why It Matters
Response bias appears on nearly every AP Statistics exam in the survey-design and experimental-design free-response sections. Beyond the exam, professionals in public health, market research, and political polling must recognize and minimize response bias to draw valid conclusions. Misjudging public opinion or health behaviors because of biased questions can lead to costly policy errors.
Common Mistakes
Mistake: Confusing response bias with nonresponse bias
Correction: Response bias is about wrong answers from people who participate. Nonresponse bias is about missing answers from people who refuse or cannot be reached. On an AP exam, clearly name which type applies.
Mistake: Assuming a large sample size eliminates response bias
Correction: Increasing the sample size reduces random sampling variability, but it does nothing to fix a systematic measurement problem. A biased question asked of 10,000 people is still biased.