Experiments and observational studies – random assignment, placebo, blocking and lurking variables
MathematicsStatisticsAges 15–16
How do you mainly use this simulation?
Use it with your class: students open it from a link or a QR code, with no account needed.
Does a memory-training app raise test scores? Put made-up students into app, look-alike app (placebo) and no-app groups by chance, by chance within blocks, by their own choice or by tapping them yourself. See how motivation, a lurking variable, confounds the observational study but is balanced by random assignment, repeat each design 200 times, and read the scope of inference: cause and effect or only an association, for the sample or for the whole school.
Lesson: Experimental design: random assignment, control and placebo groups, blocking, confounding and the scope of inference
What it shows
An experiment imposes a treatment; an observational study only records what people choose. Here made-up students try a memory-training app for four weeks, and their gain in a memory test is measured. Motivated students improve more, and when students choose for themselves the motivated ones pick the app, so motivation confounds the comparison. Random assignment balances motivation and every other lurking variable on average, a look-alike app (placebo) separates expectations from the app's own effect, and blocking by pre-test score makes the comparison more precise. The readouts state what each design allows: cause and effect needs random assignment, and generalising to the school needs a random sample.
How to use
Choose the Subjects (volunteers or a random sample of the school), the Number of subjects, the Assignment and the Groups. Press Assign again for a new random split, or with You choose tap each student to move it between groups. Press Run the study to measure the gains and record the group means and differences. Press Repeat 200 times to run the design again and again, then compare the average differences and their spread. Tick Show the lurking variable to colour each student by motivation.
Parameters you can change
- Subjects Volunteers from one class, Random sample of the school
- Number of subjects 24–60
- Groups App, look-alike app, no app, App and no app
- Assignment Random, Random within blocks (pre-test score), Students choose (observational study), You choose (tap students)
- Show the lurking variable (motivation)
Questions to explore
- Why does the app look better in the observational study than in the randomized experiment?
- What can you measure with the look-alike app group that the no-app group cannot show?
- Why are the differences from repeated studies less spread out when you block by pre-test score?