Subject

Science & Epistemology

How claims are tested, what counts as evidence, and the ways inquiry goes wrong. The most precept-heavy subject in the ledger, and the precepts are largely this ledger's own working rules: falsifiability, the burden of proof, the razors. It also holds the entries about the reliability of research itself, which is uncomfortable and belongs here.

Search within this subject · All subjects

Evidence

Claims data can bear on.

Essay: The ways evidence misleads

Leading digits in many datasets follow a logarithmic distribution; useful for fraud detection.

For datasets generated by suitable multiplicative or scale-spanning processes, about 30.1% of leading digits are 1 and about 4.6% are 9. Deviations can flag data for review, but neither conformity nor its absence proves fraud, and fabricated numbers need not be too evenly distributed.

In practice

  • An expense claim for exactly £249.99 in a file where the limit is £250 is not evidence on its own, but a file where every claim ends in 9 is worth opening.
  • An auditor sampling invoices notices too few beginning with 1 and finds a spreadsheet where the numbers were typed rather than calculated.
Attributed to
Frank Benford; earlier Simon Newcomb
Standing
Unrelated to Benford's law of controversy; itself an instance of Stigler's law of eponymy.
Further reading
Wikipedia

References on a topic concentrate in a few core journals and scatter thinly across many, in a fixed geometric ratio.

Bradford found in 1934 that to double the papers found on a topic you must consult roughly five times as many journals. It is the reason a literature search hits diminishing returns quickly and then falls off a cliff.

In practice

  • A researcher finds nine of the twelve useful papers in two journals and the last three across eleven more.
  • A support team finds most answers in three documents and spends the rest of the week in forty others.
Attributed to
Samuel C. Bradford
Standing
Completes the Lotka, Price, and Zipf family of concentration laws.

Some initially reported effects shrink in later studies.

Schooler documented it across parapsychology, ecology and medicine. It can arise from selective reporting, regression to the mean, changing methods or populations, or genuine change over time, and the mundane explanations account for most cases.

In practice

  • A pilot study shows a 40% improvement, the full trial shows 12%, and the rollout shows 4%.
  • An A/B test that produced a striking uplift produces a modest one when repeated on the whole user base.
Attributed to
Jonathan Schooler
Possible explanations
Candidate explanations are selective reporting, which publishes the largest early estimates; regression to the mean; changing methods and populations as a field matures; and genuine variation in the effect itself.
In its favour
The pattern is documented across several fields, and it predicted the replication crisis rather than following it.
Against it
The causes are disputed and mostly mundane, since selective reporting and regression account for the bulk of it, so the name can suggest a mysterious process where the explanation is ordinary.
Standing
The pattern is documented; the causes are disputed, and most cases are explained by selective reporting and regression to the mean.
Further reading
Wikipedia

A researcher's expectations leak into procedure, prompting and scoring, and move the result.

Rosenthal's 1963 demonstration used rats labelled "maze-bright" and "maze-dull" at random, and the labelled-bright rats performed better. Blinding exists because of this result.

In practice

  • An interviewer who rates a candidate highly asks easier follow-ups and scores the answers generously, without noticing either.
  • A trial assessor who knows which arm the patient is in records ambiguous symptoms differently, which is why allocation is concealed.
Attributed to
Robert Rosenthal
Also called
Rosenthal effect; observer-expectancy effect
Standing
The reason blinding exists. The classroom version is the Pygmalion effect, listed separately.

Distinct vocabulary in a corpus grows as a sublinear power of total tokens.

Herdan and Heaps independently. The practical version: a corpus ten times larger yields maybe three times the vocabulary, which is why dictionary compilation hits diminishing returns.

In practice

  • A support knowledge base of 500 articles covers most questions; at 5,000 articles it covers slightly more and is far harder to search.
  • A style guide of forty rules catches most errors; four hundred rules catch a few more and are never read.
Attributed to
Harold Stanley Heaps
Also called
Herdan's law
Further reading
Wikipedia

An explanation constructed after the fact to fit the outcome, with no way to have been wrong.

Gould and Lewontin's 1979 critique, borrowing Kipling's title. The test is whether the account could have accommodated the opposite outcome equally well — if so, it explains nothing.

In practice

  • An account of a firm's success that names its culture as the cause would have named its culture as the cause had the firm failed.
  • A retrospective that explains an outage by "insufficient testing" would have explained a clean quarter by "our testing culture".
Attributed to
Gould & Lewontin, from Kipling
Further reading
Wikipedia

For non-perishable things such as ideas and technologies, remaining life expectancy grows with age already survived.

Formalised by Mandelbrot and Taleb. A technology in use for 50 years can be expected to last another 50; one that arrived last year probably will not. It applies only to non-perishables — the reverse holds for humans.

In practice

  • A book still in print after two centuries is a safer recommendation than one that appeared in the spring.
  • A restaurant open since 1970 is a better bet for next decade than the one that opened last spring, and this reverses for the chefs.
Attributed to
Albert Goldman; formalised by Mandelbrot and Taleb
Also called
Lindy effect
Further reading
Wikipedia

A person can expect a one-in-a-million event about once a month, given the rate at which events occur.

Littlewood's arithmetic: at roughly one event per second during eight hours of alertness, a million events pass in about 35 days. Miracles are what a large denominator looks like.

In practice

  • A person who thinks of a friend just before the friend calls has had roughly a million thoughts that month.
  • An organisation that experiences one truly improbable coincidence a year is behaving exactly as a large denominator predicts.
Attributed to
J. E. Littlewood
Further reading
Wikipedia

Doctrine displacing evidence inside a scientific institution, with the cost paid outside it.

Lysenko's rejection of Mendelian genetics became Soviet state policy in 1948, after which opposing geneticists lost their posts and some their lives. The episode is the standing case for what happens when a discipline's internal correction mechanism is switched off from outside: the error held for a generation, and the agricultural cost was borne by people with no view on genetics.

In practice

  • A firm where contradicting the founder's theory ends careers accumulates evidence against it that nobody presents.
  • A field whose funding flows through one committee produces a literature shaped like that committee.
Attributed to
Trofim Lysenko
Also called
Michurinism; politically enforced doctrine
Further reading
Wikipedia

Scientists fight bitterly over priority because recognition is the field's only currency, so disputes are structural rather than personal failings.

Newton and Leibniz on calculus, Darwin and Wallace on selection. Merton's point is that the disputes are produced by the reward structure, so treating them as personal failings misdiagnoses them.

In practice

  • Two teams reach the same architecture independently and spend a quarter arguing about whose proposal it was.
  • Two departments both claim the cost saving in the annual report, and the dispute is about credit rather than about the saving.
Attributed to
Robert K. Merton
Standing
Explains why multiple discovery and Stigler's law of eponymy generate so much conflict.

Treating a property of one's own model as a property of the world, so that ignorance is read as randomness and unfamiliarity as complexity.

Jaynes introduced it in probability theory: probability describes a state of knowledge, and calling an event random asserts something about the speaker rather than the event. The general form covers a great deal of confused argument, since mysterious, obvious, simple and weird are all relations between a mind and a thing, and all get stated as though they were features of the thing.

In practice

  • A process described as unpredictable is being described by someone who cannot predict it.
  • A design called intuitive is intuitive to the person who built it, which is the one datum that does not count.
Attributed to
E. T. Jaynes; popularised by Eliezer Yudkowsky
Also called
Projecting the map onto the territory
Further reading
Wikipedia

For a fixed false-positive rate, the likelihood-ratio test is the most powerful; decision rules trade Type I against Type II error.

Neyman and Pearson's 1933 result. Its institutional consequence is that every screening programme, every audit threshold and every legal standard of proof is a choice about which error to prefer.

In practice

  • A screening programme set to catch almost every case will refer many healthy people, and the threshold is a policy choice.
  • An audit rule that flags 5% of transactions has chosen how many frauds it will miss, whether or not anyone stated it.
Attributed to
Neyman & Pearson
Further reading
Wikipedia

Assuming every observed feature exists because it is optimal, rather than as a by-product or an accident of history.

Gould and Lewontin's 1979 spandrels paper, taking the name from Voltaire's Dr Pangloss. The chin, they argued, is a by-product of jaw development rather than an adaptation for anything.

In practice

  • An organ that appears exquisitely adapted may be a by-product of the developmental process that produced its neighbour.
  • A firm's decision structure that looks perfectly designed for its market is often a residue of a merger nobody would repeat.
Attributed to
Gould & Lewontin, from Voltaire
Also called
Adaptationism

Positive and novel results are likelier to be published, so the literature overstates effects.

Sterling documented it in 1959, and the file-drawer problem is the standard name. Registered reports and pre-registration exist because of it, and they measurably reduce the positive-result rate.

In practice

  • A drug's four published trials all show benefit and its three unpublished ones do not, and the meta-analysis reads the four.
  • A company that only writes up successful experiments has an internal literature in which every initiative worked.
Attributed to
Meta-research; Theodore Sterling
Also called
File-drawer problem
Further reading
Wikipedia

Sincere witnesses to the same event produce irreconcilable accounts, so contradiction is not evidence of dishonesty.

Named from Kurosawa's 1950 film, in which four accounts of one killing are irreconcilable. It is a term of art in law, journalism and qualitative research precisely because none of the witnesses is lying.

In practice

  • Three witnesses to one collision give three different sequences, and the camera footage matches none of them exactly.
  • An incident review where four engineers describe different sequences is not evidence of a cover-up; it is what memory under stress produces.
Attributed to
Named from Kurosawa's film
Standing
Used as a term of art in law, journalism, and qualitative research.
Further reading
Wikipedia

Extreme measurements tend to be followed by more moderate ones.

Galton's 1886 finding from parents' and children's heights. It is why performance appears to improve after criticism and worsen after praise, and why almost every unbiased intervention looks effective when applied to extremes.

In practice

  • A pupil who scored 92% one week and 74% the next is described as having lost focus, and nothing about the pupil changed.
  • A store's worst-performing branch improves after intervention, and would have improved without it, and the intervention gets the credit.
Attributed to
Francis Galton
Standing
The managerial sacking bounce in sport is the best-known applied case: results improve after a dismissal about as much as for clubs that keep their manager.
Further reading
Wikipedia

Many scientific studies cannot be replicated, questioning their validity.

The Open Science Collaboration's 2015 attempt reproduced 36 of 100 psychology findings. Cancer biology and economics have shown comparable rates, so the problem is not confined to one field.

In practice

  • A textbook chapter rests on four studies, three of which have failed to reproduce, and the chapter is unchanged in the next edition.
  • A company builds a feature on a widely cited finding and cannot reproduce the effect in its own test at ten times the sample size.
Attributed to
Ioannidis; Open Science Collaboration
Also called
Reproducibility crisis
Further reading
Wikipedia

Samples are not representative of the population, skewing conclusions.

Wald's wartime work on returning bombers is the canonical case: armour belongs where the survivors were not hit. The sample was defined by having survived, which reversed the obvious reading.

In practice

  • A satisfaction survey sent to current customers cannot reach the ones who left, which is the question it was commissioned to answer.
  • A study of successful founders drawn from a list of successful founders can say nothing about what causes success.
Attributed to
Statistics and epidemiology
Further reading
Wikipedia

Focusing on successes while ignoring failures leads to false conclusions.

Wald's bomber analysis is the cleanest case, and mutual fund performance tables are the everyday one: closed funds vanish from the record, so the surviving average overstates the category.

In practice

  • A street of eighteenth-century buildings suggests they built well then, and the ones built badly are not there to be counted.
  • A management technique's success stories are collected from firms that survived, and the ones that did the same thing and failed are not available to be surveyed.
Attributed to
Statistics; Abraham Wald
Further reading
Wikipedia

Moving cases from one group to another can raise the average of both groups, so measured outcomes improve while no individual is better off.

Feinstein, Sosin and Wells named it in 1985 from cancer staging. Better scans found small metastases that older methods missed, so patients formerly counted as early stage moved into the advanced group. They were the sickest of the early group and the healthiest of the advanced one, so survival statistics improved for both while nobody lived a day longer. The name comes from Will Rogers's joke that Oklahomans moving to California raised the average intelligence of both states.

In practice

  • A school moves its weakest top-set pupils into the middle set and reports higher averages in both sets.
  • A country narrows its definition of unemployment so that discouraged workers are counted elsewhere, and two statistics improve at once.
Attributed to
Feinstein, Sosin & Wells
Also called
Stage migration
Standing
A pure artefact of reclassification, which is why comparisons over time need the classification held fixed. Compare Simpson's paradox and Goodhart's law.

Frequent citation of publications without original primary data creates false authority.

Named from Winnie-the-Pooh, where Pooh and Piglet track their own footprints and conclude a Woozle exists. Citation chains that lead back to a single unsupported source produce the same effect.

In practice

  • A figure appears in a press release, is quoted in a committee report, and is then cited back to the committee as independent corroboration.
  • A statistic cited in four papers that all trace to one unsourced 1978 estimate has acquired authority from circulation rather than from evidence.
Attributed to
Wohlwill, named from A. A. Milne
Also called
Evidence by citation; named from Winnie-the-Pooh
Further reading
Wikipedia

Rank-frequency distributions in several domains are often approximately power-law-like, with word frequencies the classic case.

Zipf's finding: the most common word appears about twice as often as the second and three times as often as the third. Exact exponents are not universal, and the city-size and firm-size applications show substantial variation and remain empirically debated.

In practice

  • A support team finds a handful of question types cover most of its volume, and a very long list covers the remainder one case at a time.
  • Concentration recurs across languages, city sizes and firm sizes; the exact exponent does not, and no agreed explanation covers the family.
Attributed to
George Kingsley Zipf
Also called
Rank-size rule
Possible explanations
One explanation is preferential attachment, in which frequent items become likelier to be used again. Another is a balance between speaker and hearer effort. A third is that simple random processes produce the same shape, which is why the pattern alone does not identify a mechanism.
In its favour
The regularity in word frequencies is strong, holds across languages and corpora, and is a standard benchmark in language modelling.
Against it
Exponents vary, deviations at both tails are common, the city and firm applications are disputed, and because random typing reproduces the shape, the law describes without explaining.
Standing
Exact exponents are not universal; city-size and firm-size applications vary substantially and remain debated.
Further reading
Wikipedia

Model

Explanatory accounts and formal models.

Essay: How knowledge grows, and how inquiry fools itself

Understanding is distorted by four standing sources of error: the idols of the tribe, the cave, the marketplace, and the theatre.

Bacon published them in 1620: the tribe is human nature's own distortions, the cave is individual temperament, the marketplace is language, the theatre is inherited doctrine. Marketplace and theatre have no clean modern equivalent, which is why the list still repays reading.

In practice

  • A department's headcount model is built on assumptions everyone in the department shares and nobody has stated — the idols of the tribe, on a spreadsheet.
  • A committee adopts the vocabulary of a consultancy and finds it can no longer describe the problem it hired the consultancy to solve.
Attributed to
Francis Bacon
Also called
The four idols; idola
Standing
The earliest systematic catalogue of cognitive bias, three and a half centuries before the experimental literature.

People can believe they hold a belief while making none of the predictions it implies, and act on the second while sincerely professing the first.

The diagnostic is anticipation again. Yudkowsky's illustration is a man who says a dragon lives in his garage and has an explanation ready for every test that would detect one, which means he is arranging his expectations around its absence while professing its presence. The state is not hypocrisy: the professing is sincere and the anticipations are separate.

In practice

  • An organisation that sincerely believes its product is best does not behave as though a competitor could not win the account.
  • A person who says the meeting could be an email attends it and schedules the next one.
Attributed to
Eliezer Yudkowsky, from Daniel Dennett
Also called
Professing without anticipating

No hypothesis is tested in isolation, so a failed prediction never tells you which assumption to abandon.

The 1919 eclipse could have been read as refuting Newton, as an error in the instruments, or as an atmospheric artefact. Which assumption to abandon is a judgement no experiment settles.

In practice

  • A dashboard shows no traffic and the team cannot tell whether the campaign failed, the tracking broke, or the definition of a session changed.
  • A drug trial misses its endpoint, and the sponsor must decide whether the drug does not work, the dose was wrong, or the population was chosen badly.
Attributed to
Duhem & Quine
Also called
Underdetermination of theory by evidence; confirmation holism
Further reading
Wikipedia

Someone who has repeatedly been convinced by arguments that turned out to be wrong may rationally stop updating on arguments they cannot refute.

Alexander's case is a reader persuaded in turn by a book and its rebuttal, who concludes that being unable to counter an argument is weak evidence about the argument. Since persuasive skill is distributed independently of truth, the non-expert's best policy may be to hold position against compelling reasoning, which is epistemically uncomfortable and often correct.

In practice

  • A juror who cannot fault either expert and declines to be moved by the more fluent one is behaving sensibly.
  • A manager who has been talked into four failed initiatives is right to distrust the fifth pitch on its persuasiveness.
Attributed to
Scott Alexander
Also called
Refusing to be convinced by good arguments

Cross-cultural cases are often not independent, because related or neighbouring societies share traits through descent, diffusion and contact.

Galton raised the objection at a meeting of the Royal Anthropological Institute in 1889, after Tylor presented correlations across hundreds of societies: if the societies inherited or borrowed the trait from one another, they are not independent observations and the sample size is a fiction. The problem has never gone away, and every modern cross-cultural study has to say how it handles it.

In practice

  • A survey of forty firms in one industry that all copied the same consultancy has one observation with thirty-nine echoes.
  • A finding replicated across twelve European countries has been replicated across one civilisation.
Attributed to
Francis Galton
Also called
Cross-cultural non-independence; cultural autocorrelation

Analytic choices made after seeing the data inflate false positives even with no intent to p-hack.

Gelman and Loken's 2013 point, and the sharper half of the p-hacking critique: a researcher who would have run a different analysis had the data differed inflates false positives without ever running two analyses.

In practice

  • An analyst tries three date windows, two exclusion rules and a log transform before the result becomes significant, and reports one analysis honestly.
  • A trial that would have been reported one way had the effect appeared in men, and another way had it appeared in women, has more paths than it declares.
Attributed to
Gelman & Loken
Also called
Researcher degrees of freedom
Further reading
Wikipedia

Under combinations of low prior odds, low power, bias and analytic flexibility, most statistically significant published claims can be false.

Ioannidis's 2005 paper is the most-downloaded in PLoS Medicine's history. The argument is arithmetic rather than cynical: he showed mathematically that given typical power, prior odds and analytic flexibility, a majority of published positives can be false.

In practice

  • A field where studies typically enrol forty participants and test six outcomes will publish a great many findings that do not survive replication.
  • A company running dozens of small A/B tests at 95% confidence ships several changes a year that do nothing.
Attributed to
John Ioannidis
Also called
Why most published research findings are false
Possible explanations
The explanation is arithmetic: with low prior odds, modest power, bias and analytic flexibility, the proportion of significant findings that are true can fall below a half, whatever the nominal error rate says.
In its favour
The derivation is transparent, the parameter values it assumes match those measured in several fields, and the replication crisis that followed found roughly the failure rates it implied.
Against it
It is a mathematical argument from assumed priors, power and bias rather than a measurement of how many published findings are false, and fields differ enough in those parameters that a single headline claim does not transfer.
Standing
A mathematical argument from assumed priors, power and bias rather than a direct measurement of how many findings are false.

Normal science accumulates anomalies, enters crisis and undergoes paradigm change; successive paradigms may be partly incommensurable.

Kuhn's 1962 book. The incommensurability claim is the contested part — that rival paradigms cannot be fully compared — and Lakatos's research programmes were written partly to answer it.

In practice

  • A field accumulates twenty anomalies filed as measurement error, and then one account explains all twenty and the vocabulary changes.
  • An engineering team explains away nine intermittent faults until someone proposes a different model of the system and all nine become one fault.
Attributed to
Thomas Kuhn
Possible explanations
The explanation is that a paradigm supplies the problems, methods and standards of a field, so anomalies accumulate without dislodging it until an alternative exists that can absorb them.
In its favour
The account of normal science, crisis and change fits several well-documented episodes, and it reshaped how science is taught and studied.
Against it
The incommensurability claim is the contested part, since rival paradigms are compared in practice all the time, and Lakatos's research programmes were written partly to supply the standards Kuhn's account left out.
Standing
The incommensurability claim is the contested part; the account of crisis and change is more widely accepted.

Theories are judged not one test at a time but by whether their research programme keeps predicting novel facts or merely absorbs anomalies.

Lakatos's 1970 framework distinguishes a programme that predicts new facts from one that only accommodates old ones after the event. It is the standard reconciliation of Popper with Kuhn.

In practice

  • A theory that predicted a new particle and then found it is progressing; one that has explained every failed prediction after the fact is not.
  • A management framework that keeps generating testable claims is worth more than one whose defenders explain every failure as poor implementation.
Attributed to
Imre Lakatos
Also called
Progressive and degenerating problemshifts
Standing
The standard reconciliation of Popper's falsificationism with Kuhn's account of how science actually proceeds.

Practical local knowledge acquired by doing is not reducible to codified technique, and schemes that ignore it fail in ways their designers cannot see.

Scott's 1998 distinction. Metis is what the local farmer knows about this field in this season; techne is the agronomy manual. Schemes that recognise only the second fail in ways their designers cannot see.

In practice

  • A national curriculum cannot encode what the teacher knows about the four pupils in this class who are stuck.
  • An agricultural scheme that specifies planting dates for a region fails on the farm where the soil drains differently, and the farmer cannot say why in the scheme's vocabulary.
Attributed to
James C. Scott
Standing
Converges independently on Hayek's knowledge problem and Polanyi's paradox.

Giving a recurring observation a stable name makes it easier to recognise, compare, retrieve and debate, and supplies no evidence that the category is coherent or the claim true.

Mill's chapter on naming as subsidiary to induction argues that names give thought a point of attachment, while explicitly denying that names are indispensable to reasoning. Lupyan, Rakison and McClelland found in 2007 that even redundant labels helped people learn novel categories, and that a nonverbal association did not. Against that sits reification: once a proposed pattern becomes a noun, sentences begin treating it as an established object. The ledger's own architecture creates the bias, which is why inclusion here is not endorsement.

In practice

  • A team that names a recurring failure can search for it, count it and argue about its boundaries, none of which it could do while the failure was only a story.
  • The same name, repeated confidently for a year, has the department certain a thing exists that nobody has yet measured.
Attributed to
Coined for this ledger; anchored in J. S. Mill
Also called
The empiriceme about empiricemes; the point of attachment
Possible explanations
The explanation is retrieval and comparison: a named pattern can be recalled, matched against a new case and argued about, where an unnamed one has to be reconstructed each time.
In its favour
Mill supplies the historical argument and experimental work on labels and categorisation supplies evidence that naming changes what people notice and remember.
Against it
It is a coinage for this ledger and a synthesis rather than a single finding, and the same property makes bad claims travel, which is the cost the principle carries with it.
Standing
A coinage for this ledger and a synthesis rather than a single finding. Mill supplies the historical anchor, Lupyan the experimental evidence on labels, Fricker the collective case and Hacking the looping caution; the predictions it generates are largely untested. Set out in full in the essay of the same name. Sailer's Law of Neologisms disputes part of it, holding that self-explanatory names are precisely the ones that fail to travel.

Perception is modelled as the brain generating predictions and processing only the error between prediction and input, rather than building percepts from the senses upward.

On this account the sensory stream mostly confirms what was already predicted, and what travels up the hierarchy is the discrepancy. It explains sensory attenuation, several illusions, and the cost of surprise economically. The unease among critics is that a framework which explains everything from perception to action to psychiatry may be doing so by being unfalsifiable.

In practice

  • A word misheard in noise is heard as the word that was expected, and the expectation arrived first.
  • A familiar route feels shorter because almost nothing along it needs processing.
Attributed to
Karl Friston; Andy Clark
Also called
Predictive coding; the Bayesian brain
Possible explanations
The explanation is that the brain maintains a generative model and passes upward only the discrepancy between prediction and input, which is cheaper than transmitting everything and explains why the expected is barely noticed.
In its favour
It accounts for a wide range of findings in one framework, including sensory attenuation, several illusions and aspects of attention, and it has generated testable predictions in perception research.
Against it
Its scope is the main criticism, since a framework that accommodates almost any result is hard to refute, and critics question whether its central formalism is doing explanatory work or being fitted after the fact.
Standing
An ambitious unifying framework whose scope is its main criticism: it accommodates a very wide range of findings, and critics question what observation could count against it.

No number of past regularities logically guarantees the next instance; induction cannot be justified by induction.

Hume's 1739 problem. Distinct from Hume's razor, which concerns proportioning causes to effects. No amount of past regularity licenses the next case, and no argument from experience can establish that it does.

In practice

  • A bridge inspected annually for eighty years is inspected again, because the eighty years are a reason to expect and not a reason to know.
  • A supplier with a four-year perfect record has given you no logical guarantee about next month, which is why contracts specify remedies rather than trust.
Attributed to
David Hume
Standing
Distinct from Hume's razor, which concerns proportioning causes to effects.

Continuous adaptation is necessary just to maintain current fitness levels.

Van Valen's 1973 hypothesis, named from Alice: it takes all the running you can do to stay in the same place. Sexual reproduction and the arms race with parasites are the biological cases.

In practice

  • A firm that stops improving its product does not stand still; it falls behind because everyone else improved.
  • A profession whose qualification was sufficient in 1995 requires continuous study now to hold the same relative standing.
Attributed to
Leigh Van Valen
Also called
Red Queen's race, from Through the Looking-Glass
Further reading
Wikipedia

Some answers end inquiry without explaining anything, because they are shaped like explanations and carry no predictions.

Yudkowsky's examples are words that arrive at the end of a causal chain and stop it: God, democracy, the free market, human nature. The test is not whether the word is true but whether anything follows from it. A stopsign feels like a floor and functions as a door that nobody opens.

In practice

  • An incident explained by human error has been labelled rather than investigated.
  • A decision justified by the culture here has been closed to the only question that would have improved it.
Attributed to
Eliezer Yudkowsky
Also called
Curiosity-stopper; the thought-terminating answer
Standing
Compare the maxim that naming is not explaining, and the entry on Mill's Handle, which names the same risk from the other direction.

If the conclusion was fixed before the arguments were assembled, the arguments carry no evidence whatever the reader thinks of their quality.

Yudkowsky's device is a clerk who writes the verdict at the bottom of the page and then fills in the reasons above it. However good the reasons, the page's accuracy was determined when the bottom line was written, and reading the argument tells you nothing about the world. It is the sharpest available statement of why motivated reasoning cannot be corrected by better arguments.

In practice

  • A report commissioned to support a decision already taken is not evidence for it, however rigorous.
  • A researcher who knows which result would be publishable has written part of the bottom line already.
Attributed to
Eliezer Yudkowsky
Also called
Writing the conclusion first

Concepts change within a discipline by variation and selection rather than by wholesale replacement.

Toulmin's 1972 alternative to Kuhn's revolutions. He argued disciplines change as populations of concepts do — gradually, with variation and selection — rather than by wholesale paradigm replacement.

In practice

  • A profession that adopted a word for a procedure in 1985 uses the same word in 2020 for something the 1985 practitioners would not recognise.
  • A discipline's vocabulary in 1990 and 2020 shares most of its terms with drifted meanings, which is variation and selection rather than replacement.
Attributed to
Stephen Toulmin
Standing
Advanced as the evolutionary alternative to Kuhn's revolutionary account.

The historical record overstates the prevalence of disorder, because what is written down is what was exceptional enough to record.

Tuchman proposed in A Distant Mirror that the persistence of an event in the record makes it appear more common than it was, and suggested discounting accordingly. The mechanism is a selection effect on the archive rather than an error by any chronicler: ordinary years generate no documents, so a period reconstructed from documents is a period reconstructed from its emergencies.

In practice

  • A safety review reading only the incident reports is reading the years when something happened.
  • A century known chiefly for its wars had decades nobody wrote down.
Attributed to
Barbara Tuchman
Also called
The overreporting of disaster
Standing
Formulated by a historian about historical sources, and the same selection operates on news, incident logs and any archive built from what was worth reporting.

Precept

Statements of how one ought to act or design.

Essay: Rules for weighing claims

If something cannot be settled by experiment or observation, it is not worth debating.

Alder proposed it against philosophical disputes that generate no observable difference. It is deliberately sharper than Occam's razor, which merely prefers the simpler account.

In practice

  • An argument about whether the team is "truly agile" produces no observable difference in what anyone does on Monday.
  • Two philosophers dispute whether a thermostat "wants" the room warm; no experiment separates the answers, so the dispute is about vocabulary.
Attributed to
Mike Alder
Also called
Newton's Flaming Laser Sword
Standing
Frequently misspelled as Adler's razor.

Every model simplifies and so is wrong in some respect; the practical question is whether it is wrong in ways that matter for its purpose.

Box first wrote that all models are wrong in a 1976 paper on science and statistics, arguing that a correct model cannot be reached by ever more elaboration and that scientists should look for economical ones instead. The full phrase, adding that some are useful, appeared in his writing from 1978 and in his 1987 book with Norman Draper, which also put the practical question: how wrong a model has to be before it stops being useful.

In practice

  • An epidemiologist estimates how fast an outbreak will grow from a simple model that ignores who meets whom, because it answers the question being asked.
  • A bank keeps its credit-risk model after a crisis exposes its assumptions, but limits how much weight any single decision may rest on it.
Attributed to
George Box
Also called
All models are wrong, but some are useful; Box's aphorism
Possible explanations
The explanation is that a model is a simplification by design: leaving things out is what makes it usable, so being wrong in detail is the price of being useful. The question is then whether it is wrong in the ways that matter for the purpose at hand.
In its favour
It is standard teaching in statistics and modelling, and practice bears it out: large forecasting competitions have repeatedly found simple methods competing with complex ones, and simple models are easier to check.
Against it
Critics note that the aphorism can excuse a poor model as merely one more wrong one, and that it offers no way to tell a useful model from a misleading one. Some statisticians also object that calling approximations wrong blurs the difference between simplification and error.
Standing
The wording developed across Box's writing from 1976 to 1987. Compare Better Vaguely Right Than Exactly Wrong and the Map–Territory Relation.
Further reading
Wikipedia

A checklist for claim evaluation: independent confirmation, genuine debate, quantification, falsifiability, and Occam's razor.

From Sagan's 1995 The Demon-Haunted World. Its practical value is as a sequence: most bad claims fail at independent confirmation before the harder tests are needed.

In practice

  • An investment claim is checked for independent confirmation first, which disposes of it before anyone has to evaluate the returns model.
  • A health claim survives quantification and falsifiability and fails at debate, because no critic has ever been allowed to test it.
Attributed to
Carl Sagan

In subjects that do not allow precision, a rough answer that is right is worth more than an exact answer that is wrong.

Read wrote it in his textbook Logic: Deductive and Inductive (1898), in a chapter on the terms of ordinary language, warning that in reasoning upon some subjects it is a mistake to aim at an unattainable precision. The line is now usually credited to Keynes, often in the form roughly right rather than precisely wrong; quotation researchers first find it attached to him in an accountants' magazine in 1966, twenty years after his death.

In practice

  • A council planning school places works from a population forecast with a wide range, rather than a single figure from a model whose assumptions it cannot check.
  • A doctor tells a patient that recovery usually takes two to six weeks, rather than naming a date.
Attributed to
Carveth Read
Also called
Roughly right rather than precisely wrong; Better approximately right than precisely wrong
Possible explanations
One explanation is that precision without accuracy invites false confidence: a figure given to the decimal place looks measured whether or not it is. Another is decision value, since a roughly right estimate points towards the right action while an exact but mistaken one points confidently the wrong way.
In its favour
Consistent with forecasting research, where calibrated ranges outperform confident point estimates, and with the practice of reporting uncertainty in science and engineering.
Against it
A rival adage holds the opposite, that it is better to be definitely wrong than vaguely right, because a definite claim can be tested and corrected; the idea goes back to Bacon's remark that truth emerges more readily from error than from confusion. Read's maxim gives no test for when vagueness has become evasion.
Standing
Usually misattributed to Keynes. Compare All Models Are Wrong, which makes the same trade from the modeller's side.

The obligation to produce evidence rests on whoever asserts, not on whoever doubts.

The claimant produces evidence; the doubter need not disprove. Reversing it — demanding proof that something is not so — creates an obligation that cannot be discharged.

In practice

  • Someone claiming a colleague leaked the document owes evidence; the colleague owes nothing until it arrives.
  • A vendor asserting their system is secure must show it, and "no one has proved it insecure" is not the same sentence.
Attributed to
Roman law
Also called
Ei incumbit probatio qui dicit; onus probandi

Independent evidence from different sources can converge on the same conclusion.

Whewell coined the term in 1840 for the convergence of independent inductions. Evolution is the standard case: fossils, comparative anatomy, biogeography and genetics converge without depending on each other.

In practice

  • A fraud case rests on the phone records, the travel bookings and the accounting entries, none of which depends on the others.
  • A medical diagnosis is trusted when the blood test, the imaging and the symptom history point the same way through independent routes.
Attributed to
William Whewell
Also called
Convergence of evidence

There is something wrong with everything, there is a literature on everything, and all propositions about real interest rates are wrong.

Cowen set the three out on Marginal Revolution, having coined them while teaching macroeconomics. He notes the common thread himself: each is an instruction not to place too much confidence in your own judgement. The first is the load-bearing one, since it holds that until you have found the major flaws in an argument you do not yet understand it, which makes locating the weakness a condition of comprehension rather than a stage after it.

In practice

  • A reviewer who cannot state the strongest objection to a paper has not finished reading it.
  • A team certain no one has studied its problem has not searched, which is what the second law predicts.
Attributed to
Tyler Cowen
Also called
There is something wrong with everything; there is a literature on everything

Scientific honesty requires reporting everything that might show you wrong, not only what supports you.

Feynman's 1974 Caltech address. His example was Millikan's oil-drop experiment, whose successors crept toward the true value slowly because nobody wanted to report a number far from Millikan's.

In practice

  • A researcher reports the three analyses that did not support the finding alongside the one that did.
  • An engineer's design review lists the two failure modes they could not rule out, which is the section reviewers read first.
Attributed to
Richard Feynman
Also called
Cargo cult science address

Patients can have as many diseases as they well please.

The clinical counterweight to Occam's razor. In elderly patients multiple simultaneous conditions are the norm, so parsimony is the wrong prior and unifying diagnoses are often wrong.

In practice

  • An elderly patient's breathlessness is heart failure and anaemia and a chest infection, and looking for one diagnosis delays two treatments.
  • A production outage has a slow query, an undersized cache and a retry storm, and fixing only the query returns it to service until Thursday.
Attributed to
John Hickam
Standing
The clinical counterweight to Occam's razor.
Further reading
Wikipedia

Do not explain behaviour by higher mental faculties if lower ones suffice.

Morgan's 1894 canon, still the default rule in comparative psychology. It is Occam's razor applied to minds, and it is why attributing planning to an animal requires ruling out simpler accounts first.

In practice

  • A dog that returns to where food was is not planning, and the simpler account should be excluded before the richer one is offered.
  • A system that appears to anticipate user needs is usually pattern-matching, and attributing intent to it changes how it is trusted.
Attributed to
C. Lloyd Morgan
Standing
Occam's razor applied to minds; still the default rule in comparative psychology.
Further reading
Wikipedia

A representation is never the thing represented, and confusing the two is the standard failure of models and metrics.

Korzybski's 1931 formulation. Borges's one-to-one map is the reductio: a map as detailed as the territory is useless, so every map is an omission, and the question is always which omission.

In practice

  • A risk model that reports 3.2% is not reporting a property of the world; it is reporting a property of the model.
  • A performance rating of 4.1 is a number about a form, and treating it as a number about a person is where appraisal systems fail.
Attributed to
Alfred Korzybski
Also called
The map is not the territory; Borges's 1:1 map
Further reading
Wikipedia

A resource arrangement that works in practice can work in theory; theory should be revised to fit observed institutions.

Ostrom's formulation is a rebuke to theory that dismisses what it cannot model. Working arrangements are data, and a theory that says they cannot work is the thing requiring revision.

In practice

  • A theory that says a commons must be enclosed or nationalised meets a Swiss alpine pasture managed communally since 1517, and the theory is what needs revising.
  • A model that says an arrangement cannot work, applied to an arrangement that has worked for decades, is data about the model.
Attributed to
Elinor Ostrom

The whole meaning of a concept is the sum of its conceivable practical effects; where there is no difference in practice there is no difference in meaning.

Peirce's 1878 maxim founded pragmatism. Its practical use is as a dissolver: if two positions predict the same observations in every case, the dispute between them may be verbal.

In practice

  • Two positions that predict identical observations in every case may be disagreeing about vocabulary rather than about the world.
  • A dispute over whether a team is "really" agile that produces no different action on Monday has been dissolved rather than settled.
Attributed to
Charles Sanders Peirce

Falsifiability was proposed as the criterion demarcating empirical science from non-science.

Popper's 1934 criterion. Its use as a demarcation line is contested, with Lakatos's research programmes and the Duhem-Quine thesis both complicating it, and it survives best as a discipline for stating claims that could fail.

In practice

  • A consultant whose framework explains every outcome after the fact has offered nothing that could have been wrong beforehand.
  • It survives as a discipline rather than a test: a claim stated so that it could fail is more useful than one that could not, whatever its scientific status.
Attributed to
Karl Popper
Also called
Law of falsifiability; criterion of demarcation
Possible explanations
The explanation is asymmetry: no number of confirmations establishes a universal claim, while one clear refutation overturns it, so the capacity to be refuted is what gives a theory empirical content.
In its favour
It remains an excellent working heuristic, it names what is wrong with theories that explain every outcome, and it survives in practice as the demand that a claim specify what would count against it.
Against it
As a demarcation line it is contested: the Duhem-Quine thesis shows that a refutation can always be deflected onto auxiliary assumptions, and Lakatos's research programmes describe what scientists actually do when an anomaly appears.
Standing
Its standing as a demarcation criterion is contested; Lakatos and the Duhem-Quine thesis both complicate it.

Where a field adopts the appearance of measurement without the underlying constancy, its confidence outruns its warrant.

Hayek's 1974 Nobel lecture. His target was macroeconomics adopting the form of physics without its constancies, and the argument applies to any field where measurement outpaces theory.

In practice

  • A model with fourteen estimated parameters forecasts next year's demand to the nearest thousand units, and has never come within 20%.
  • A performance management system that produces a score to one decimal place has adopted the appearance of measurement without the underlying stability.
Attributed to
Friedrich Hayek
Also called
Scientism critique

Everything must have a reason or cause.

Leibniz's principle. Its cost is that it licenses demanding an explanation where none exists, which is why it sits uneasily with quantum mechanics and with genuinely stochastic processes.

In practice

  • A quality manager asks why a batch failed and gets a defensible answer; asks why that batch and not the eleven identical ones, and gets a story.
  • A post-incident review that insists every anomaly had a cause will manufacture one, which is where the root cause fallacy comes from.
Attributed to
Gottfried Wilhelm Leibniz
Further reading
Wikipedia

Extraordinary claims require extraordinary evidence.

Sagan's formulation, with earlier versions in Laplace and Hume. Its practical use is to make the required evidence proportional to the prior, which is Bayesian updating stated as a slogan.

In practice

  • A colleague reporting that the lift is broken is believed; a colleague reporting that the lift was sabotaged is asked how they know.
  • A claim that a supplier missed one delivery needs an email; a claim that they are defrauding you needs considerably more.
Attributed to
Carl Sagan; earlier Laplace and Hume
Also called
Extraordinary claims require extraordinary evidence

Natural laws operate the same everywhere and at all times.

Hutton and Lyell's principle, and the necessary assumption behind all historical science. Without it no inference from present processes to past events is licensed.

In practice

  • A geologist dates a canyon from erosion rates measured this decade, and the whole figure rests on the rate having held for six million years.
  • An auditor projects a fraud pattern back over seven years of records on the assumption that the accounting system behaved then as it does now.
Attributed to
James Hutton; Charles Lyell
Also called
Uniformitarianism

Figure

Sayings, images and puzzles.

Essay: Paradoxes in the data, and sayings about knowing

When the public rallies to an idea that distinguished experts reject, the experts are usually right.

Asimov offered it in a 1977 essay as a deliberately sceptical counterweight to Clarke's first law, which he thought encouraged credulity, and especially against popular pseudoscientific enthusiasms. It is an aphorism rather than an empirical estimate of how often experts or crowds are right.

In practice

  • A viral claim that a common additive causes harm is rejected by every toxicologist who has looked at it, and the toxicologists are right.
  • A widely shared theory about an aircraft crash is dismissed by accident investigators, and the final report vindicates them.
Attributed to
Isaac Asimov
Standing
The counterweight to Clarke's first law, which Asimov wrote it against.

Selection bias in observational studies can create spurious negative correlations between variables.

If a hospital admits people for either of two unrelated conditions, the two conditions will appear negatively correlated among inpatients. It is a standing hazard in any study drawn from a selected population.

In practice

  • Among people already in hospital, smokers appear to have fewer of a second condition — because you need to be sicker in other ways to be admitted as a non-smoker.
  • A firm finds its most technically able hires are its worst communicators, because it hired anyone strong on either.
Attributed to
Joseph Berkson
Standing
The birth-weight paradox and the obesity paradox are the best known medical cases.
Further reading
Wikipedia

Among low-birth-weight babies, those born to smokers have lower infant mortality than those born to non-smokers, although smoking raises mortality overall.

Yerushalmy reported the pattern in 1971, and it was read for decades as evidence that smoking somehow protected small babies. The resolution is selection. Restricting the comparison to low-birth-weight babies compares infants who are small for a known and relatively mild reason with infants who are small for unknown and often graver ones. Hernandez-Diaz, Schisterman and Hernan set it out in 2006 as the textbook case of collider stratification bias.

In practice

  • A study of hospital patients finds smoking protective, because it compares only people already ill enough to be admitted.
  • A firm finds that test scores predict nothing among its hires, because the low scorers who got through had something else going for them.
Attributed to
Jacob Yerushalmy
Also called
Low birth-weight paradox
Standing
Settled rather than open: the pattern is an artefact of conditioning on a variable that both causes affect. Compare Berkson's paradox and the obesity paradox.
Further reading
Wikipedia

For any doctrine one wishes to believe, there is never a shortage of arguments with which to support it.

Kolakowski's observation, and the epistemic complement of motivated reasoning: the supply of arguments is never the binding constraint on belief, so argument alone rarely changes anyone's mind.

In practice

  • A board that has decided to acquire finds four independent justifications by Thursday, and would have found four others for declining.
  • A person who wants the job finds the salary, the commute and the culture all point one way, and would have found the reverse had they not wanted it.
Attributed to
Leszek Kolakowski
Standing
The epistemic complement of motivated reasoning.

Two defensible statistical analyses of the same data can reach opposite conclusions about a group difference, and the data alone cannot say which is right.

Lord posed it in 1967. A university weighs every student in September and June. One statistician compares the change in weight and finds that men and women gained the same on average, so the diet had no differential effect. A second adjusts each student's June weight for their September weight and finds a difference. Both calculations are correct. Choosing between them requires a causal assumption that the numbers do not contain, which is why Pearl later treated the puzzle as a problem of causal diagrams rather than of statistics.

In practice

  • A hospital and its regulator publish opposite conclusions about the same ward protocol, one comparing before and after, the other adjusting for baseline severity.
  • Two colleagues argue about whether a training course helped, without noticing that they are asking different questions of the same spreadsheet.
Attributed to
Frederic Lord
Standing
Not a flaw in either method. An adjustment encodes a causal claim, and the data cannot choose the claim for you. Compare Simpson's paradox and Berkson's paradox.

Among patients with certain chronic diseases, those who are overweight survive longer than those of normal weight, although excess weight raises the risk of developing the disease.

Reported in dialysis and heart failure patients in the late 1990s and named in cardiology soon after. The explanations divide: selection, since the comparison begins only after diagnosis; reverse causation, since illness itself causes weight loss; confounding by smoking, which lowers weight and raises mortality; and the crudeness of body mass index as a measure of fat. Studies that exclude smokers and patients with recent weight loss tend to weaken or remove the effect.

In practice

  • A registry reports better survival for heavier heart failure patients, and the advantage disappears once those who lost weight in the previous year are excluded.
  • A clinician advises a patient to lose weight while the registry on his desk seems to say otherwise, because the registry counts only people who already have the disease.
Attributed to
Clinical epidemiology; named in cardiology
Also called
Reverse epidemiology
Possible explanations
Candidate explanations are selection, since the comparison begins after diagnosis; reverse causation, since illness itself causes weight loss; confounding by smoking, which lowers weight and raises mortality; and the crudeness of body mass index as a measure of fat.
In its favour
The pattern in the data is not in doubt, and it has been reported repeatedly across heart failure, dialysis and other chronic conditions.
Against it
Studies that exclude smokers and patients with recent weight loss weaken or remove it, which is what an artefact of conditioning on diagnosis would predict.
Standing
Contested. The pattern in the data is not in doubt; whether it is a fact about bodies or an artefact of conditioning on diagnosis is. Compare the birth-weight paradox, its structural twin.

Evolution is cleverer than you are.

Orgel's rule from molecular biology. Its general form is a warning against declaring a feature useless: the absence of a known function is evidence about the observer, not about the system.

In practice

  • A protein that looks like a design error turns out to be doing three things at once.
  • A legacy process that appears redundant is holding together two systems nobody has mapped, which is Chesterton's fence in biological form.
Attributed to
Leslie Orgel

The organ reporting that it is remarkable is the one under assessment, so introspective testimony about the mind cannot be independent of it.

Philips's line observes that his high opinion of the brain was arriving from the brain. The joke states a real constraint: self-report about cognition is generated by the system being reported on, so a person's confidence in their own reasoning is not evidence about that reasoning, and the research on confabulation shows the report is produced whether or not there is anything to report.

In practice

  • A candidate's account of how they reach decisions is a decision-making product, not an observation of one.
  • A team confident in its own objectivity has produced that judgement using the faculty in question.
Attributed to
Emo Philips
Also called
Look what is telling you that
Standing
A folk statement of the problem introspection faces as evidence; compare the illusion of explanatory depth and confabulation research.

A new scientific truth prevails not by convincing its opponents but by outliving them.

Planck wrote it in his scientific autobiography. Its empirical test came in 2019: Azoulay and colleagues found that after a star scientist's death, publications by outsiders in that field rose measurably.

In practice

  • A journal that would not publish a method while one editor sat on the board publishes four papers using it in the year after he steps down.
  • A department's methodological orthodoxy shifts within eighteen months of one retirement, having been immovable for a decade.
Attributed to
Max Planck
Also called
Science advances one funeral at a time
Further reading
Wikipedia

People mistake mediated representations for reality, and resist correction because the representations are all they have known.

Plato's Republic, book seven. The part usually skipped is the ending: the returning prisoner is not thanked but ridiculed, and Plato suggests they would kill him.

In practice

  • A returning secondee explains how the other office does it and is told, kindly, that things are different here.
  • An analyst who reports that the dashboard everyone trusts is measuring the wrong thing is treated as the problem.
Attributed to
Plato
Also called
Allegory of the cave

We know more than we can tell; tacit knowledge cannot be fully articulated.

Polanyi's 1966 formulation: we know how to ride a bicycle or recognise a face without being able to state the rules. It was the standing limit on rule-based automation until statistical learning bypassed it.

In practice

  • A radiographer who spots the anomaly cannot say what drew their eye, which is why the training is apprenticeship rather than a checklist.
  • A chef's "until it looks right" is not evasion; the knowledge exists and does not survive being written down.
Attributed to
Michael Polanyi
Also called
Tacit knowledge
Further reading
Wikipedia

Forcing the phenomenon to fit the measure or the model, and discarding whatever will not fit.

From Greek myth: Procrustes stretched or amputated guests to fit the bed. It is the mythic ancestor of the McNamara fallacy, and Taleb revived the term for exactly that use.

In practice

  • A patient whose symptoms match no diagnostic code is recorded under the nearest one, and the statistics know only the nearest one.
  • A reporting template with five categories forces every project into one of five shapes, and the projects reorganise themselves to fit.
Attributed to
Greek myth; revived by Taleb
Standing
The mythic ancestor of the McNamara fallacy.

A trend present in every subgroup can reverse or vanish when the subgroups are aggregated.

Berkeley's 1973 graduate admissions data showed an aggregate disadvantage for women. Department-level analysis found few statistically significant differences and about as many departments favouring women as men; pooled properly, the data showed a small significant bias in favour of women. The aggregate pattern arose largely because women applied more often to harder-to-enter departments.

In practice

  • A drug that does better than the alternative in men and better in women can look worse overall, if the trial enrolled them in different proportions.
  • A company's overall pay gap can be large while every role pays equally, because the roles are not equally staffed — and the aggregate figure is the one that matters politically.
Attributed to
Edward Simpson; earlier Udny Yule
Also called
Yule–Simpson effect; amalgamation paradox
Standing
Compare Lord's paradox, where two correct analyses of the same data disagree, and the Will Rogers phenomenon, where reclassification alone moves both averages.

In theory there is no difference between theory and practice; in practice there is.

The self-swallowing structure is the point: the claim that theory and practice agree is itself a piece of theory, and the exception it admits is the whole of the difficulty. What it names is the gap between a model's stated assumptions and the conditions under which it is actually applied, which is where most failures of applied knowledge occur.

In practice

  • A process that works on the whiteboard meets the one supplier who will not use the format.
  • A model validated on clean data meets the data the organisation actually keeps.
Attributed to
Attributed to Jan van de Snepscheut and to Yogi Berra
Also called
In theory there is no difference
Possible explanations
The explanation is that theory omits what it takes to be irrelevant, and practice is where the omitted details turn out to matter, so the gap is built into the act of modelling.
In its favour
The self-swallowing structure makes the point efficiently, and the observation is confirmed daily in engineering, medicine and software.
Against it
It is a joke rather than a claim, offers no account of which omissions will matter, and its attribution is disputed between several sources and established for none.
Standing
Attribution is disputed between several sources and is not established for any of them.

Any figure that looks interesting is probably wrong.

Twyman's law is the analyst's discipline. A striking number is far more often an error in collection, definition or joins than a genuine discovery, so surprise should trigger checking before publishing.

In practice

  • A dashboard showing signups up 400% overnight is a duplicated feed four times out of five.
  • An analyst who publishes a remarkable figure without checking the pipeline has usually published a bug.
Attributed to
Tony Twyman
Further reading
Wikipedia