Curriculum project outside-view prediction

Curriculum project outside-view prediction is the forecasting episode in which Daniel Kahneman and a team designing a high-school curriculum discovered that their confident estimate of a manageable two-year project was contradicted by the history of similar undertakings. Its decisive significance lies in the reversal: direct familiarity with the project produced an optimistic story, while comparable cases revealed a much longer and riskier distribution of outcomes. The team recognized the warning, yet continued anyway, eventually completing the textbook after eight years, when the Israeli Ministry of Education no longer used it.

The Project and Its Initial Forecast

A few years after Daniel Kahneman began collaborating with Amos Tversky, he persuaded officials in the Israeli Ministry of Education to support a curriculum for teaching judgment and decision making in high schools. The design team combined experienced teachers, psychology students, and Seymour Fox, dean of the Hebrew University’s School of Education and an authority on curriculum development. For about a year, the group met every Friday afternoon, created a detailed syllabus, wrote several chapters, and tested sample lessons in classrooms. Their visible progress made the project feel concrete and controllable.

When the team discussed methods for estimating uncertain quantities, Kahneman asked each member to submit a confidential estimate of how long it would take to deliver a finished draft to the Ministry. The private collection of judgments was itself deliberate: gathering estimates separately was intended to use the group’s knowledge more effectively than beginning with an open discussion. The answers clustered narrowly around two years, ranging from one and a half to two and a half years. The consensus was not merely optimistic; it was made persuasive by the team’s immediate experience of having already written chapters and developed a workable plan.

The Outside View Appears

Kahneman then asked Seymour Fox to identify teams that had previously developed comparable curricula from scratch, including projects undertaken during the period of pedagogical innovations such as new math. Fox knew several of their histories and was asked to consider them at the same stage of progress as Kahneman’s group. The question transformed the forecast from an account of the team’s own plan into a comparison with a reference class.

Fox’s answer undermined the consensus. He realized that a substantial fraction of comparable teams had never completed their work and estimated the failure rate at about 40 percent. Among the teams that did finish, he could recall none that took less than seven years or more than ten. When Kahneman asked how the current team compared with those earlier groups, Fox judged it below average, though not by much. The judgment surprised everyone, including Fox, because the knowledge supporting it had not entered his initial estimate. Only the explicit reference to comparable cases connected his historical knowledge with the project’s future.

The new information was not difficult to understand, but it was difficult to feel. The team could see a reasonable plan and imagine a book arriving in roughly two years; it could not imagine the strange sequence of obstacles that might stretch the work across a decade. The statistics should have prompted a causal inference: if many similar teams failed and successful teams required at least seven years, curriculum writing was probably harder than the project’s internal plan suggested. Instead, the group treated the base-rate information as pallid and set it aside because it conflicted with the vivid impression of its own progress.

Failure to Act on the Warning

The evidence gave the team a strong reason to stop. No one was willing to invest six additional years in a project carrying a 40 percent chance of failure, yet the danger did not produce an immediate, emotionally compelling command to quit. After a few minutes of desultory debate, the members collected themselves and continued as though the disturbing forecast had not been delivered.

The eventual result validated the outside view far more closely than the original estimate. The book was completed eight years later, after many unpredictable vicissitudes. By then, Kahneman was no longer living in Israel and had long since ceased to participate. The Ministry of Education’s initial enthusiasm had waned, and the completed text was never used. The outcome therefore combined delay, organizational attrition, and practical failure: the manuscript existed, but its institutional purpose had disappeared by the time it arrived.

Kahneman later described the decision to continue as irrational perseverance. The group had recognized that persistence was not reasonable, but surrendered rationality rather than surrendering the enterprise. This distinction makes the episode more than a mistaken estimate. The forecasting failure became a decision failure because the team did not allow its newly acquired evidence to change its commitment.

Inside View, Planning Fallacy, and Method

The inside view is the spontaneous approach the team first used. It concentrates on the particular project, its available resources, the chapters already drafted, and the time previously required to write them. Even a cautious member tends to begin with the team’s own plan and add only a modest margin for error. Because the plan is specific and imaginable, it exerts more psychological force than statistics describing a broad population of comparable projects.

The outside view begins elsewhere. It asks what happened to similar projects and uses their distribution of outcomes as the starting point for prediction. Kahneman and Amos Tversky called the tendency to produce forecasts unrealistically close to best-case scenarios the planning fallacy. The curriculum project supplied its central personal demonstration: the team’s two-year estimate was not a neutral calculation but an optimistic best-case scenario shaped by the inside view.

The proposed remedy is reference class forecasting. Planners first identify an appropriate class of comparable undertakings, such as kitchen renovations or large railway projects, then consult a broad database containing plans and actual outcomes. Such a database can reveal likely cost and time overruns, failure rates, and underperformance for different project types. Only after establishing that external baseline should decision makers adjust the forecast for clearly supported differences between the current project and its reference class.

This method also assigns responsibility beyond the planners who create an unrealistic schedule. Some plans are made optimistic because approval depends on persuading superiors or clients, while projects are rarely abandoned merely because they exceed their budgets or deadlines. Decision makers who approve plans must therefore insist on distributional information. Ignoring the outside view at the approval stage is itself a form of the planning fallacy.

Wider Meaning for Prediction

The curriculum episode sits within a broader critique of predictive expertise. Philip Tetlock’s twenty-year study interviewed 284 people who earned their living commenting on political and economic trends and collected more than 80,000 probability judgments. The experts performed worse than an equal-probability baseline across three possible outcomes, and even specialists were not significantly better than nonspecialists in their own regions of expertise. Knowledge improved forecasts only slightly, while extensive knowledge could intensify the illusion of skill and overconfidence.

The same calibration problem appears when judges construct confidence intervals. Properly calibrated intervals should contain the true values at the stated rate, yet many subjects produce intervals that are too narrow; in most studies, actual values fall outside the relevant extreme bounds in roughly 30 percent of cases. Likewise, a Duke survey of 11,600 forecasts by corporate chief financial officers found that their estimates of the following year’s Standard & Poor’s index returns had a correlation with actual returns slightly below zero. Their 80 percent confidence intervals contained far too many surprises: 67 percent of outcomes fell outside the expected range.

These comparisons clarify what the curriculum team failed to do. It did not lack intelligence, information, or commitment. It lacked a disciplined way to place its vivid local evidence beside the less vivid but more diagnostic record of comparable cases. The episode’s lasting lesson is methodological and psychological at once: a plausible plan can be less informative than a distribution of past outcomes, and recognizing that truth does not guarantee that people will act on it.