Empiriceme · Evidence · Bias / Effect

John Henry Effect

Also called Compensatory rivalry

A control group that knows it is being compared with a new method works harder to vindicate the old one, masking the new method's effect.

Saretsky named it in 1972, after the steel-driver of the ballad who died beating a steam drill, from a federal experiment in performance contracting in which teachers in the comparison classrooms, aware that they were being measured against outside contractors, produced unusually good results. Cook and Campbell later listed it among the threats to validity as compensatory rivalry.

In practice

  • A hospital trials a new discharge process on two wards, and the comparison wards, stung at being the old way, cut their own delays and make the trial look like a draw.
  • A salesman told that a new colleague's method is being tested against his own has the best month of his year.
Standing
Documented mainly in case reports rather than measured across studies. The mirror image of the Hawthorne effect, which works on the group receiving the change.
Subject
Psychology & Behavior
Essay
A century and a half of measuring the mind

Search every entry·Every entry, A to Z