AP Statistics — collecting data

Unit 3 of AP Statistics is the one unit where the arithmetic is almost absent and the reasoning decides everything. Ten questions here, each on a separate idea, in the order the course builds them: which number in a described study is the parameter and which the statistic; what randomising a sample buys that enlarging a convenience sample never will; how stratified and cluster sampling are actually carried out, and why strata are meant to be internally similar while clusters are meant to resemble the whole; when systematic sampling fails, which is when the interval lines up with a rhythm already in the population; and the three kinds of bias that survive a perfectly random design — undercoverage, non-response and response bias, which are three different failures and are regularly confused for one.

The second half turns to experiments. What separates an observational study from an experiment is that someone imposed a treatment, not that the study was large or careful — a point one question is built entirely around. The three principles then get one question of their own, with comparison, random assignment and replication each doing a job the other two cannot: a control group to compare against, random assignment to balance the variables nobody thought to measure, and enough subjects that ordinary variation does not masquerade as an effect. Blinding follows, with the trap named openly — blinding does not stop the placebo effect; the inactive pill exists so that the effect operates in both groups and can be subtracted out. Confounding is asked as a critic's objection to a real-sounding association. The last question is the one the whole unit exists for: a sample drawn at random from a single school's roll, then randomly assigned to two teaching methods, licenses a causal claim and licenses generalisation no further than that one school.

Every study, researcher and institution in the bank is invented, and each question says so. Nothing is taken from any College Board publication, released examination, course and exam description or scoring guideline; the contexts and the numbers are all made up here, and what is being practised is the reasoning the exam asks for rather than any exam's own material. There are no images, graphs or tables — every study is described in words, which means the questions can be answered anywhere, though it also means the graph-reading that the real Unit 3 questions sometimes require is not practised here.

Zestly is an independent study tool. It is not affiliated with the College Board, which owns the AP Statistics exam, and it is not an exam centre.

• Say which figure in a described study is a parameter and which is a statistic
• Explain what random selection buys, and why a bigger convenience sample does not buy it
• Tell stratified from cluster sampling by how each is carried out, and say when each is the sensible choice
• Name the circumstance in which systematic sampling produces a systematically unrepresentative sample
• Distinguish undercoverage, non-response and response bias, and say who each one leaves out or misrepresents
• Decide whether a described study is observational or an experiment, on the imposition of a treatment rather than on its size
• State what comparison, random assignment and replication each accomplish separately
• Say what single and double blinding protect against, and why an inactive pill is given to the control group
• Identify a confounding variable in a described association and explain why more data would not remove it
• Work out what a study licenses: causation from random assignment, and generalisation only to the population actually sampled

In an invented study, a researcher draws fifty pupils at random from the roll of one large secondary school, then assigns each of them at random to a new teaching method or to the existing one. What may she conclude from a difference in test scores?