Leaderless group challenge

The Leaderless group challenge was an Israeli Army selection exercise in which observers watched strangers struggle together with a log and a wall, then converted their impressions of leadership into predictions about officer training and combat. The exercise seemed to make character visible in action, yet repeated feedback showed that the resulting forecasts were barely better than guesses. Its importance is therefore not simply as a military test, but as a compact demonstration of how coherent impressions can survive overwhelming evidence of their own failure.

Identity and Setting

The exercise belonged to methods developed by the British Army during World War II and was used in the Israeli Army to help evaluate candidates for officer training. Daniel Kahneman encountered it during his national service after completing an undergraduate degree in psychology and serving for a year as an infantry officer before being assigned to the army’s Psychology Branch. Under a scorching sun, he and a colleague watched groups of sweaty soldiers attempt the task and recorded their conduct under pressure.

Eight candidates, strangers to one another, entered the obstacle field with every insignia of rank removed. They were identified only by numbered tags, a design that stripped away formal hierarchy before the test began. Their assignment was to lift a long log from the ground and carry it to a wall about six feet high. The entire group had to cross to the other side without the log touching either the ground or the wall and without any candidate touching the wall. Any violation required the group to declare the attempt a failure and begin again.

Course of the Challenge

The task allowed several solutions but made cooperation unavoidable. In one common method, some candidates crossed by crawling over the pole while others held it at an angle like a giant fishing rod. In another, soldiers climbed onto one another’s shoulders and jumped across. The final candidate faced the most precarious maneuver: while the others kept the log suspended, he had to jump toward it, move along its length, and leap safely over the wall. Failure at this stage sent the entire group back to the beginning, turning fatigue and frustration into further tests of collective discipline.

The observers treated the repeated attempts as a behavioral display. They noted who took charge, who attempted to lead but was rebuffed, and who contributed steadily to the group effort. They also watched for stubbornness, submissiveness, arrogance, patience, temper, persistence, and quitting. A rejected idea could produce competitive spite, with its proposer withdrawing effort; a mistake could provoke one soldier to berate another; and an exhausted restart could reveal who stepped forward to lead. The artificial obstacle course therefore appeared to offer a direct view of each candidate’s nature.

Judgments and Predictive Failure

After several attempts, the observers summarized each soldier’s leadership ability and assigned a numerical score determining eligibility for officer training. Their impressions felt so immediate and coherent that the task of evaluation seemed easy. Some candidates appeared strong, others weak, arrogant, mediocre, or beyond consideration for officer rank. A soldier who took control when the group was in trouble and led it over the wall seemed, by the obvious logic of the scene, likely to become an effective leader in training and combat.

The confidence of the assessments was strikingly specific. Observers readily concluded that one candidate would never succeed, another would do adequately, and another would become a star. They were willing to acknowledge in general terms that anything could happen, but that abstract qualification did not weaken their definite judgments about particular individuals. The observed behavior had formed a compelling story, and the story seemed to point directly toward the future.

The feedback was consistently discouraging. Every few months, the evaluators learned how the cadets were performing at officer-training school and compared those results with the original assessments and with the judgments of commanders who had observed the cadets over a longer period. The pattern remained the same: the evaluators’ ability to predict later performance was negligible. Their forecasts were better than blind guesses, but not by much.

The Illusion of Validity

The crucial failure was not merely that the predictions were inaccurate. The global evidence of repeated predictive failure should have reduced confidence in the next assessment and encouraged more moderate forecasts, but it did neither. The evaluators knew that their judgments were little better than random guesses while continuing to feel that each new candidate’s character had been plainly revealed. The next group was brought to the obstacle field, the log was lifted, and the same certainty returned within minutes.

Kahneman compared this persistence to the Müller-Lyer illusion: people can know that two lines are equal in length and still see them as different. The observers knew that their selection method had failed, yet the immediate impression of validity remained intact. He named this experience the illusion of validity and identified it as his first cognitive illusion.

The challenge also became a model for several broader mechanisms. A brief performance in an artificial situation was substituted for the more difficult question of how a candidate would perform over months of training or in combat. The observers relied on the representativeness heuristic, treating behavior that looked like leadership as evidence of stable leadership ability. They made nonregressive predictions of outstanding success or certain failure from weak evidence, while WYSIATI ensured that the behavior visible during the exercise dominated their judgment and obscured everything they did not know about the candidate’s future.

Broader Significance

The exercise illustrates why confidence can be generated by coherence rather than by evidence. Once the observers’ separate impressions converged into a persuasive account, the quality and quantity of the evidence mattered less than the smoothness of the story. The obstacle field supplied vivid actions, visible crises, and emotionally legible roles; it did not supply a reliable basis for forecasting a long and complex career. Its dramatic clarity became a source of error.

The same illusion later appeared in professional prediction. Philip Tetlock’s twenty-year study of 284 people who earned their living commenting on political and economic trends collected more than 80,000 forecasts and found that the experts performed worse than an equal-probability baseline across three possible outcomes. Even in areas they knew best, specialists were not significantly better than nonspecialists; greater knowledge often strengthened the illusion of skill, while famous forecasters tended to make more flamboyant and overconfident predictions.

The pattern also appears in quantitative forecasting. A Duke University survey collected 11,600 forecasts from corporate chief financial officers about the next year’s Standard & Poor’s index returns. The correlation between their estimates and actual returns was slightly less than zero, and their supposedly 80% confidence intervals contained far too many surprises: 67% of outcomes fell outside the stated range, rather than the expected 20%. Like the observers at the wall, the CFOs constructed coherent estimates from information that came readily to mind while failing to represent what they did not know.

The Leaderless group challenge thus stands as a formative scene in the study of judgment. It does not show that behavior is meaningless or that observation has no value. It shows that a compelling performance can invite a confidence disproportionate to its evidential reach, and that awareness of aggregate failure may not correct the force of a particular story. The log and wall made leadership feel visible; the later records revealed how little that visibility could predict.