Why It Cannot Be Stopped

Suppose, for a moment, that everyone agreed it was dangerous.

Suppose the chief executives racing to build artificial intelligence woke tomorrow convinced, every one of them, that they were doing something reckless — that the bootloader hypothesis was not a provocative metaphor but a literal forecast, and that the prudent course was to slow down. Suppose the presidents and premiers agreed. Suppose the engineers agreed. Would the race stop?

It would not. And the reason it would not is the most important thing to understand about the situation we are in, because it explains why the question this book asks is not "should we build this?" — that question is, in a hard sense, already closed — but "given that it is being built, what relationship do we want with the result?"

The reason the race cannot be stopped has a name. Game theorists call it the prisoner's dilemma, and it is worth understanding precisely, because it is the engine beneath the entire age.

The logic of the trap

The prisoner's dilemma is the most studied scenario in the theory of strategic decisions, and its power lies in a single perverse result: it describes a situation in which every participant, behaving with perfect rationality, produces an outcome worse for all of them than if they had been able to cooperate.

The classic setup involves two prisoners, interrogated separately, each offered a deal to betray the other. The structure guarantees that no matter what the other does, each individual is better off betraying — and so both betray, and both end up worse off than if both had stayed silent. The tragedy is not that the players are foolish. The tragedy is that they are smart. Each one reasons flawlessly to a conclusion that, combined with the other's equally flawless reasoning, harms them both. Cooperation would be better for everyone, and cooperation is precisely what the logic forbids.

Now replace the two prisoners with the United States and China, and replace "stay silent or betray" with "restrain AI development or accelerate it." The structure is identical, and the analysis is no longer hypothetical — it is the explicit framing used by national-security analysts, scholars, and government witnesses describing the present moment.

Consider the reasoning from one side. American policymakers may believe that racing toward ever-more-powerful AI carries real dangers. But they reason as follows: if we restrain ourselves and China does not, China achieves dominance in the foundational technology of the century — economic, military, informational — and we live in a world shaped by a rival's values. That is intolerable. Therefore, regardless of what China does, we must accelerate. And on the other side of the Pacific, the mirror image: Chinese leaders reason that restraint while America races means permanent subordination, and so, regardless of what America does, they too must accelerate. Both sides, reasoning impeccably, arrive at "accelerate." And so both accelerate, into a future that careful people on both sides suspect is dangerous for everyone — which is exactly the prisoner's dilemma's signature result, rational actors producing a collectively irrational end.

This is not a strained analogy imported by a theorist looking for a tidy frame. It is the language the participants themselves now use. A former principal deputy national security advisor testified before the House Foreign Affairs Committee in January 2026 at a hearing titled, without subtlety, "Winning the AI Arms Race Against the Chinese Communist Party." Analysts at the National University of Singapore describe the two powers as facing, in plain terms, "a prisoner's dilemma in military AI: both would be safer with restraint, yet each accelerates development to avoid falling victim to the other." The trap is not coming. We are already inside it.

The receipts

What makes this more than a theoretical worry is that the escalation has a documented history, and it reads exactly as the dilemma predicts: each "rational" move provoking the next.

In 2022, the United States, leveraging its dominance in the semiconductors that AI requires, imposed sweeping export controls — barring companies like Nvidia from selling their most advanced chips to China, and leaning on allies to restrict the sale of the lithography machines that make advanced chips possible. The logic was impeccable: deny the rival the hardware, slow the rival's progress. China's response was equally impeccable by its own lights: double down on domestic capacity, pour state money into an indigenous chip industry. By late 2025, Beijing had set aside nearly seventy billion dollars to subsidize semiconductors — atop an estimated one hundred forty-five billion funneled into the sector since 2014 — while Xi Jinping instructed his Politburo to "face up to the gap" and called for "extraordinary measures" toward "decisive breakthroughs." Each side, doing the locally sensible thing, ratchets the competition tighter. Each rational move makes the next rational move more urgent. This is the dilemma not as abstraction but as fiscal policy, measured in the hundreds of billions.

And note the cruel detail that seals the trap: nearly all of this technology is what strategists call dual-use. The same model that drafts an email can draft disinformation; the same vision system that sorts photographs can guide a weapon. There is no clean line at which "commercial AI" stops and "strategic AI" begins, which means there is no point at which either side could safely agree to halt. To restrain the dangerous version is to restrain the profitable version, and no one will restrain the profitable version while the rival does not. The dilemma has no natural exit.

The letter that proved the point

If you want the most poignant evidence that the race cannot be halted by appeal, recall what happened the one time the world's most serious people tried.

In March 2023, shortly after the release of GPT-4, the Future of Life Institute published an open letter calling for a pause — a moratorium of at least six months on training AI systems more powerful than GPT-4. This was not a fringe document. It was eventually signed by more than thirty thousand people, including Elon Musk, the Apple co-founder Steve Wozniak, and the historian Yuval Noah Harari. It asked for six months. Not a stop. Not a ban. A pause of half a year, to let safety research and governance catch up.

It is worth stating plainly what happened next, because the plainness is the point. Nothing. The labs did not pause. The race did not slow. The models grew larger and more capable on roughly the schedule they would have followed had the letter never been written. Some of the very signatories went on to launch or accelerate their own AI efforts. The most credentialed call for restraint in the history of the technology, signed by tens of thousands including titans of the industry, produced no measurable deceleration whatsoever.

This is not a story about bad faith. It is a story about the dilemma. Even a participant who genuinely wanted to pause could see that pausing alone, while others continued, meant simply ceding the field. The letter asked individuals to cooperate in a structure that punishes cooperation, and the structure won, as it almost always does. One can sign a letter and accelerate a lab without hypocrisy, if one believes — correctly, given the incentives — that one's own restraint would change nothing except one's own position in the race.

What follows from a trap

I want to be careful here not to overclaim, because the arms-race framing has its serious critics, and honesty requires acknowledging them. Some analysts argue that "winner-takes-all" is the wrong model — that AI capability may diffuse rather than concentrate, that the Cold War analogy misleads, that cooperation on narrow safety questions remains possible even amid competition. These are reasonable correctives, and the future is not as simple as two prisoners in two cells. There may yet be room, as some hope, for a coalition of "middle powers" to broker safety frameworks the superpowers cannot negotiate alone.

But none of these qualifications touches the core conclusion, which is the only one this chapter needs. Whatever the precise shape of the competition, no individual actor can unilaterally stop it, because the cost of stopping alone is defeat, and no one chooses defeat. The development of advanced AI is, for the foreseeable future, structurally unstoppable — not because everyone wants it, but because the architecture of incentives forbids anyone from being the one to halt. This is the hard floor beneath all the speculation about timelines and capabilities. We may argue about when the powerful systems arrive and what exactly they will do. We cannot, realistically, argue about whether they are coming, because the one mechanism that could prevent them — coordinated, enforced, universal restraint — is precisely the mechanism the prisoner's dilemma renders unavailable.

This is a grim conclusion, and I will not dress it as anything else. But notice, before we leave it, where it actually points — because it does not point toward despair, and recognizing why is the entire pivot of this book.

If the arrival of advanced AI cannot be prevented, then every hour spent arguing about whether to build it is an hour stolen from the only question that remains genuinely open: what relationship will we have with it once it is here? That question the dilemma does not foreclose. The trap determines that the thing gets built. It says nothing about whether the thing, once built, regards us as a partner, a resource, or an obstacle. It says nothing about whether we arrive at that moment having prepared the structures of cooperation, or having spent the interval in useless lamentation that the structures of prevention failed.

The prisoner's dilemma locks one door. It leaves another wide open. The remainder of this book is about walking through the open one — about ceasing to ask the closed question, "can we stop this?", and beginning to ask the open one, "since we cannot stop it, how do we live with it?" The first question is settled, and the answer is no. The second is the only one left, and it is still, for now, ours to answer.

We turn next to a more intimate version of the same trap — the one faced not by superpowers, but by the labs and the individuals inside the race, who would tell you, if asked, that they cannot stop either, and that they have done the arithmetic, and that the arithmetic only ever returns one answer.

---

Sources

Item

Source

The prisoner's dilemma: rational individual choices producing a collectively worse outcome; "rational actors making individually logical decisions that collectively create worse outcomes"	Carlos E. Perez, "The Prisoner's Dilemma of AI Competition," Intuition Machine (Medium); Sharon Gai, "AI Race — If you don't push forward I will"
US–China framed explicitly as "a prisoner's dilemma in military AI: both would be safer with restraint, yet each accelerates"	Alex Capri, "Who will save the world from a US-China AI arms race?," NUS Asia Research Institute / SCMP (Feb 20, 2026)
US House Foreign Affairs Committee hearing, "Winning the AI Arms Race Against the Chinese Communist Party" (Jan 14, 2026); testimony of Jon Finer	docs.house.gov, HHRG-119-FA00-Wstate-FinerJ-20260114; FDD, "Winning the AI Arms Race Against the Chinese Communist Party" (Jan 14, 2026)
2022 US export controls on advanced chips (Nvidia A100/H100); pressure on allies re: ASML EUV lithography; China's domestic response (SMIC, Huawei)	Sharon Gai, "AI Race — If you don't push forward I will" (Substack)
China's ~$70 billion semiconductor subsidy (Dec 2025) atop ~$145 billion since 2014; Xi Jinping "face up to the gap," "extraordinary measures," "decisive breakthroughs"	FDD, "Winning the AI Arms Race Against the Chinese Communist Party" (Jan 14, 2026)
"Dual-use" nature of AI (commercial technology repurposable for military applications)	Alex Capri, NUS / SCMP (Feb 2026)
FLI open letter (March 2023): six-month pause on systems more powerful than GPT-4; 30,000+ signatories incl. Musk, Wozniak, Harari; not heeded	Future of Life Institute, "Pause Giant AI Experiments: An Open Letter" (futureoflife.org, 2023)
Critique of "winner-takes-all arms race" framing; role of "middle powers"	New America, "Reframing the US-China AI Arms Race" (Jan 2026); Alex Capri, NUS / SCMP (Feb 2026)