Positive regard for a thing recruits the halo effect, which generates further positive attributions, which raise the regard again.
The mechanism is a loop rather than a bias: once a person or idea is held in high regard, each new fact about it is interpreted favourably, and the favourable interpretation raises the regard that produced it. Yudkowsky's proposed brake is deliberately holding one criticism in mind, since the spiral is broken by any point at which the object is allowed to be ordinary.
In practice
- A founder praised for one good decision is credited with the next four, two of which were somebody else's.
- A methodology whose advocates can name no weakness has passed the point where evidence about it is being processed.
- Possible explanations
- The proposed mechanism is a loop: high regard recruits the halo effect, each new fact is read favourably, and the favourable reading raises the regard again, with no step that corrects it.
- In its favour
- The halo effect underneath it is well documented, and the runaway pattern is recognisable in reputations that keep rising past the evidence.
- Against it
- It is a coinage argued from examples rather than measured, and no study has shown the loop running away rather than settling, so the escalation is asserted rather than demonstrated.
- Standing
- A coinage of the rationalist community, argued from examples rather than tested. The halo effect underneath it is well documented; the runaway feedback version is an extension. Compare evaporative cooling of group beliefs, which describes what happens to the dissenters.
- Subject
- Psychology & Behavior
- Essay
- Theories of the mind, from the lab to the bookshop