A pattern defined after the data are seen will always be found, and proves nothing.
Named from the marksman firing at a barn and painting the target around the holes. It is why pre-registration matters — a hypothesis specified after the data is not a hypothesis at all.
In practice
- An analyst runs eleven segment breakdowns, finds one significant result, and writes the report as though that segment had been the question.
- A hypothesis specified after the data is not a hypothesis, which is why pre-registration changes the false-positive rate rather than the honesty of researchers.
- Further reading
- Wikipedia
- Subject
- Decision-making & Reasoning
- Essay
- The catalogue of predictable errors