People expect small samples to resemble the parent population and over-read patterns in them.
Tversky and Kahneman's 1971 paper, aimed at researchers rather than laypeople: psychologists were over-reading small-sample results and expecting replication that the statistics did not support.
In practice
- A team concludes a change worked from eleven users, and eleven users would show that pattern by chance about a fifth of the time.
- A manager judges a hiring source from three hires, and three is not a sample.