Teaching interventions ranked by averaged effect size, with a threshold below which an intervention is held not to be worth doing.
Visible Learning synthesised hundreds of meta-analyses into a single ranked list with a hinge point around 0.4. Critics including Snook and colleagues and Bergeron have argued that the averaging combines studies whose effect sizes are not comparable, and that some computations do not survive checking. The influence of the list is out of proportion to the confidence its method supports.
In practice
- A school adopts the top three interventions on a list and drops the four beneath a threshold nobody can derive.
- An effect size averaged across reading tests and behaviour scales is a number without a referent.
- Possible explanations
- One explanation for its reach is that it converts hundreds of studies into a single ranked list, which is what a busy school system wants. The hinge point gives an apparently simple rule for what to adopt.
- In its favour
- It drew attention to practices with good support, including feedback and teacher clarity, and it pushed schools to ask what the evidence says rather than what is fashionable.
- Against it
- Averaging effect sizes across incommensurable studies and outcomes is heavily criticised, statistical errors have been identified in the underlying syntheses, and the 0.4 threshold has no principled basis.
- Standing
- Heavily criticised on method: averaging effect sizes across incommensurable studies and outcome measures, and statistical errors identified in the underlying syntheses. The rankings are widely used in policy regardless.
- Subject
- Learning & Education
- Essay
- Theories of teaching, and what schools are for