Essay · Decision-making & Reasoning · Model

Why reasoning works the way it does

The evidence group lists how judgement goes wrong. The models here try to explain why, and they divide over a question with large consequences: are the errors signs of a flawed mind, or the reasonable behaviour of a limited one?

Two pictures of the reasoning mind

The first picture begins with Herbert Simon. Bounded Rationality holds that people satisfice, choosing the first option that is good enough, because attention and information are scarce. The Adaptive Decision-Maker adds that people switch strategies according to what a task will reward. Fast-and-Frugal Heuristics push the argument furthest: simple rules that ignore information can beat complex models where the environment is uncertain. On this view, a shortcut is not a failure of reasoning but a sensible use of it.

The second picture is Dual-Process Theory, popularised as fast and slow thinking. Reasoning runs on a quick automatic system and a slow deliberate one, and most errors come from the fast system answering questions the slow one should have taken. On this view, the shortcuts are useful most of the time and predictably wrong in particular conditions, which is why they can be exploited.

Why smart people reason badly

A second cluster explains errors that intelligence does not cure. Motivated Reasoning steers thinking towards a preferred conclusion while it still feels objective. Naive Realism is the belief that one sees reality directly, so anyone who disagrees must be uninformed, irrational or biased. Cached Thoughts observes that much of what feels like thinking is retrieving a conclusion reached earlier. Near and Far Thinking adds that distance in time or probability makes thinking more abstract and idealistic. The Valley of Bad Rationality suggests that learning a little formal reasoning can make someone worse before it makes them better.

Deciding under pressure and uncertainty

A third set concerns action. Clausewitz's Fog of War insists that decisions are made on information that is incomplete, late and partly wrong. Boyd's OODA Loop argues that the advantage goes to whoever cycles through observing, orienting, deciding and acting faster. Hirschman's Hiding Hand makes the provocative claim that ignorance of difficulties lets ambitious projects start at all, and that people then rise to problems they would never have agreed to face. The Tyranny of Small Decisions shows sensible individual choices adding up to an outcome nobody chose.

Where it applies

The choice between the two pictures of the mind shapes real policy. If errors are predictable failures of the fast system, then Nudge and choice architecture can steer people towards better outcomes without restricting options. If heuristics are adaptive, the better policy is to teach people to recognise which environments their shortcuts suit. Military and business planners use the OODA loop and the fog of war as working frames, and project sponsors argue about the hiding hand whenever a large project is proposed.

Where it is argued over

The disagreement between Kahneman's and Gigerenzer's schools is the main live dispute, and it is partly a disagreement about what counts as the right answer. Dual-process theory is criticised for relabelling a continuum as two systems. Nudge interventions show heterogeneous effects once publication bias is accounted for. Flyvbjerg and Sunstein reviewed the evidence on the hiding hand and argued that its malevolent version, in which ignorance leads to disaster rather than ingenuity, is the more common outcome. The Trolley Problem is criticised for measuring willingness to accept an artificial premise rather than moral judgement.

Why it is worth knowing

These models turn a list of mistakes into an account of a mind. That matters because the two accounts recommend different remedies. One says to redesign the situation so the fast system is not misled; the other says to trust good shortcuts in the environments they were made for. Knowing both makes it possible to ask, of any bias in the evidence group, which kind of error it is, and so which kind of fix is likely to work.

The entries behind this essay: Model in Decision-making & Reasoning. Every subject has its essays on the essays page.