SAT Math — study design, bias and what a study can conclude

SAT Math includes a skill that looks more like careful reading than calculation: evaluating statistical claims about observational studies and experiments. College Board describes it in three parts. Given a study with random sampling, decide which population the results can be extended to. Given a study with or without random assignment, decide whether there is evidence of cause and effect. And understand the problems of sampling methods, and why a result reaches only the population the sample was drawn from. This material trains exactly those judgments, in twelve short situations drawn from elections, health, farming, business and local government.

The quiz opens with the two questions every study has to answer. In the first, voters drawn at random from a registration list are then randomly assigned to receive a text reminder, so the result is both causal and general. In the second, a random sample of residents reports gardening and stress, so the result is general but only an association. A third study adds self-selection: drivers who chose a safety course had fewer accidents, which tells you nothing about whether the course caused it.

Next come the tools of a good experiment: the control group that gives a fertilizer trial its baseline, and the placebo and blinding that keep expectations from being mistaken for the effect of a medication. A hospital comparison shows how a confounding variable, here the severity of injuries, can create a difference with no causal meaning at all.

The second half is about bad samples, each with its own mechanism: a radio call-in poll whose callers chose themselves, a mailed survey that only a fraction of households returned, interviews at a bus station about transit use, and a survey question worded to push people toward one answer. You also choose the one method that really produces a random sample of a company's employees, and decide between an online poll of 50,000 readers and a random sample of 900, which is where many students trust the size of a sample over the way it was chosen.

Every explanation says why each wrong answer fails, and names the concept behind the right one. The flashcards give short definitions of random selection and random assignment, what each combination of the two allows you to conclude, observational studies, confounding variables, control groups, placebos, blinding, and the main kinds of sampling bias.

The material offers a quiz and flashcards. It is independent practice; the studies are invented, and no question is taken from any published test.

• Decide which population a study's results can be extended to from how the sample was chosen
• Decide whether a study supports a cause-and-effect conclusion from whether treatments were randomly assigned
• Identify a confounding variable in an observational comparison
• Explain the role of a control group, a placebo and blinding in an experiment
• Recognize voluntary response, nonresponse, convenience-sample and question-wording bias
• Choose a method that produces a random sample and judge a large biased sample against a smaller random one

Practice material written by Zestly, based on the SAT Math skill "Evaluating statistical claims: observational studies and experiments" in the Problem-Solving and Data Analysis domain, as described in College Board's Assessment Framework for the Digital SAT Suite (version 3.01, August 2024, Appendix B). All data sets and situations are invented. Zestly is not affiliated with College Board.