illusion of validity

The illusion of validity is the tendency to experience a coherent judgment as valid even when the evidence supporting it is inadequate. It is not merely ignorance of uncertainty: people may know, in general, that their predictions are weak and still feel that each individual prediction is trustworthy. The illusion arises because System 1 rapidly constructs a persuasive account from the evidence at hand, while System 2 often accepts the account’s coherence as a substitute for testing its predictive validity.

Definition and Cognitive Basis

System 1 is designed to jump to conclusions from little evidence, but it is not designed to register the size of the jump. Under WYSIATI, only the information currently available enters the mental picture. Confidence therefore reflects the coherence and cognitive ease of the story built by System 1 and System 2, rather than the amount or quality of the evidence behind it. Poor evidence can support an attractive explanation, and some deeply held beliefs have no direct evidence beyond the fact that trusted people hold them.

This produces a crucial distinction between confidence and accuracy. Subjective confidence is a feeling, not a reasoned estimate of the probability that a judgment is correct. An admission of uncertainty can be informative, but a declaration of high confidence mainly indicates that a coherent story has been constructed. It does not establish that the story is true, nor that the underlying environment is predictable.

The Soldier-Prediction Experience

The term arose from an experience predicting which military candidates would succeed in officer training. After watching each candidate for about an hour in an artificial obstacle-field exercise, evaluators felt that the candidates’ true natures had been revealed. They made confident predictions about future performance in officer training and combat leadership, even though the observations were brief and the situation was remote from the future outcomes being predicted.

Repeated feedback showed that the predictions were only slightly better than blind guesses. Every few months, the evaluators compared their assessments with the later judgments of commanders who had observed the cadets over a much longer period. The result was consistently discouraging: the predictive ability of the original evaluations was negligible. Yet the knowledge of this general failure did not alter the next round of judgments. When another group of candidates arrived, the evaluators again watched them face the wall and lift the log, again felt that individual character had become visible, and again made confident predictions.

The failure was remarkable because the evaluators knew both things at once: their overall method had little predictive value, and each particular impression still felt valid. The experience resembled the Müller-Lyer illusion: even when people know that the lines are equal in length, they continue to see them as unequal. The analogy led to the name illusion of validity and became a model for a broader pattern in judgment. Weak evidence generated vivid impressions, and vivid impressions concealed ignorance about the many factors that would later determine performance.

Experts, Algorithms, and Low-Validity Environments

The same illusion appears in professional prediction. Philip Tetlock’s twenty-year study of 284 people who earned their living commenting on political and economic trends collected more than 80,000 probability judgments. The experts forecast events in regions they knew well and in regions about which they knew less, while also explaining how they used evidence and responded to being wrong. Their results were devastating: they performed worse than a simple procedure that assigned equal probabilities to three possible outcomes. Even in their areas of specialization, they were not significantly better than nonspecialists.

Additional knowledge helped only slightly, and the most knowledgeable forecasters were often less reliable because knowledge strengthened the illusion of skill and encouraged overconfidence. Famous forecasters made more flamboyant predictions and were more overconfident than colleagues working outside the limelight. The problem was not simply lack of training. The more fundamental question was whether the world being predicted was sufficiently regular for expert judgment to have much value.

Research comparing clinical judgment with statistical formulas reached a similar conclusion. Paul Meehl reviewed twenty studies in which trained professionals made predictions from interviews and extensive case information while formulas used only a few scores or ratings. In one example, counselors interviewed freshmen for forty-five minutes and examined grades, aptitude tests, and a personal statement, yet a formula using high-school grades and one aptitude test outperformed eleven of fourteen counselors. Across roughly two hundred later comparisons, algorithms were significantly more accurate in about 60 percent of studies; the remaining studies were ties, which still favored formulas because they were cheaper and more consistent.

These findings covered medical prognosis, hospital stays, cardiac diagnosis, sudden infant death, credit risk, business success, foster-parent suitability, recidivism, violent behavior, football results, and Bordeaux wine prices. Such domains are low-validity environments: outcomes contain enough uncertainty and unpredictability that human impressions cannot reliably extract the decisive structure. Simple formulas often outperform experts because experts try to add complexity, consider intricate combinations of features, and override rules with information that feels relevant but usually reduces validity. The broken-leg rule identifies the rare exception: a decisive new fact, such as learning that a person broke a leg, may properly override a formula predicting whether that person will go to the movies.

Inconsistency and Overconfidence

Human judgment is also unstable. Experienced radiologists assessing the same chest X-ray as normal or abnormal contradicted themselves 20 percent of the time when they saw the image on separate occasions. A comparable inconsistency appeared among 101 auditors evaluating the reliability of internal corporate audits, and a review of 41 studies found similar unreliability among auditors, pathologists, psychologists, organizational managers, and other professionals. A judgment that changes when the same information is presented again cannot be a dependable predictor.

This inconsistency reflects the context sensitivity of System 1. Unnoticed environmental influences, priming, mood, and ordinary changes in circumstances can alter a summary judgment. Even the timing of a food break affected the parole decisions of Israeli judges. Formulas do not share this instability: given the same inputs, they return the same result. Yet people often feel that their momentary impressions contain insight rather than noise, which helps preserve the illusion of validity.

The same pattern appears in financial forecasting. A Duke University survey collected 11,600 forecasts from chief financial officers estimating the next year’s Standard & Poor’s index returns. The correlation between their estimates and actual returns was slightly below zero; when they predicted a decline, the market was slightly more likely than not to rise. Worse, the CFOs did not appear to recognize that their forecasts were worthless. Their stated 80 percent confidence intervals produced surprises in 67 percent of cases, more than three times the expected rate of 20 percent. WYSIATI made the estimates feel coherent by excluding information that was unavailable, forgotten, or never known.

Calibration, Framing, and Practical Significance

The illusion extends beyond single forecasts to stated probability distributions. When judges estimate values such as the Dow Jones average at different confidence levels, proper calibration would require the actual values to fall within the corresponding intervals at the stated frequencies. In practice, actual outcomes fall outside the extreme bounds in about 30 percent of problems rather than the expected 2 percent. The resulting intervals are systematically too narrow, revealing more certainty than the judges’ knowledge warrants. Anchoring contributes to the error: people begin with a best estimate and make insufficient upward and downward adjustments.

Decision theory exposes a related conflict between psychological confidence and rational consistency. Dominance requires preferring an option that is at least as good in every respect and better in at least one; invariance requires equivalent prospects to receive the same preference regardless of how they are described. Yet framing effects repeatedly violate invariance. People prefer a sure gain when a public-health choice is described as lives saved, but prefer the equivalent gamble when the same choice is described as lives lost. The inconsistency remains even among sophisticated respondents and even after they compare their conflicting answers.

The practical lesson is not that every intuition is useless, but that confidence cannot certify validity. Predictions should be tested against outcomes, calibrated over many cases, compared with simple formulas and equal-weight baselines, and evaluated in light of the predictability of the environment. Equivalent decisions should also be deliberately reframed to reveal unstable preferences. The illusion of validity survives because coherent stories are psychologically compelling; disciplined measurement is required to determine whether those stories deserve belief.