When tech leaders trade numbers for the odds that advanced AI ends civilization, they are invoking "p(doom)"—a single probability estimate that has become Silicon Valley shorthand for existential risk. But the number itself is the least useful part of the exercise. Researchers at Georgetown's Center for Security and Emerging Technology argue that probability is often the wrong tool for AI risk, and the U.S. National Institute of Standards and Technology offers a process-oriented alternative focused on managing risk rather than guessing at it. Understanding these three approaches—and why experts disagree so sharply—is the practical core of the AI risk debate.
What is p(doom)? Understanding AI's Existential Risk
P(doom) is, as Forbes Australia reports, a Bayesian probability estimate that the development of AGI or superintelligence within a 20-to-100-year window will result in civilisational catastrophe, expressed on a percentile scale from 0 to 100. The term has become commonplace in innovation circles, with numerous tech heavyweights publicly weighing in—though Bill Gates, for one, has not offered his own value, instead suggesting that frontier AI companies may not pull the trigger themselves but have created the ecosystem and tools that could facilitate mass destruction by others.
The definition embeds several contestable assumptions: that "AGI" is a well-defined threshold, that catastrophe arrives within a specified time window, and that a single number can capture the outcome. Notably, when researchers survey experts, they must first pin down what AGI even means—one survey defined it as AI systems better at STEM research than the best human scientists, potentially with other advanced capabilities. That definitional fragility is the first hint that a single probability is doing a lot of quiet work.
The Limitations of a Single Probability: Why Experts Disagree
Why do experts disagree so widely on p(doom)? A survey of AI experts by Severin Field, conducted through a Cambridge ERA:AI fellowship, frames the question directly, noting that leading AI labs and scientists have called for the global prioritization of AI safety, citing existential risks comparable to nuclear war. The disagreement is not merely rhetorical. There is little data or detailed theory to assess AI's potentially catastrophic risks, as CSET's Andrew Lohn observes, so policymakers often resort to expert best guesses.
This creates a compounding problem. When the evidence base is thin, individual estimates reflect prior beliefs and familiarity with AI safety concepts more than measured frequencies. A single number compresses all of that—definitional choices, time horizons, causal assumptions, and personal exposure to the safety literature—into one figure that looks rigorous but obscures its own foundations. Two experts with identical "5%" answers may be describing entirely different risks.





