Sam Altman, born-again CEO of OpenAI and notorious candle in the wind, was quoted recently in the Wall Street Journal:
“We’ve been roughly right on technological predictions and pretty wrong on the social and economic implications…. …our industry underestimated how much we’re going to be able to keep people at the center of everything.”
Less than a week later, the latest in an emergent pattern of SF-based, AI-pilled intellectuals writing long, narrative (often negative) predictions on what’s to come for AI was published: AI2040.
Presented as the successor to AI2027, it’s a detailed future scenario and policy recommendation in the form of an interactive paper/website from the AI Futures Project.
Almost all of these predictive narratives, however, ignore what I refer to as the “messy middle,” where reality actually tends to unfold. The scenarios typically assume intelligence alone stands between now and the future, so more of it gets us there faster. But in the messy middle, intelligence meets the friction of the real world: institutions, supply chains, physical constraints, and the public.
Disregarding the messy middle is largely why many of the social and economic predictions Altman mentions about AI have thus far been misguided: it’s difficult to predict complexity.
“I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. Then I joined the real world and actually tried to do things…You cannot take over the world with tokens.”
The Messy Middle
Consider what’s already happened to many of the headline predictions from ~12 months ago. Confident near-term predictions of the past year took contingent, middle-heavy situations and sold them as clean conclusions.
OpenAI itself was created (as a non-profit, no less) because its founders feared that Demis Hassabis (AI good guy Demis Hassabis), would become an “AGI dictator.” Google developed the Transformer, but OpenAI figured out what to do with it.
For several years thereafter, OpenAI was the clear winner, and then Anthropic (founded by an OpenAI defector) released Claude Code and ate its lunch. Now Anthropic is back on its heels after clashing with the government and a disastrous public rollout of Fable 5, and OpenAI is back in action (now as a for-profit entity) with GPT-5.6 Sol.
Alex Karp went on CNBC and excoriated the frontier labs over “data sovereignty,” “zero data retention” and their “stealing” critical IP from their users based on their prompts and context. Amidst this backlash, both companies are scrambling as they prepare for IPOs while their customers begin evaluating the ROI of their token spend and delegate more tasks to cheaper, good-enough open weights models.
The Wall Street Journal reports OpenAI is even considering offering the US Government a ~5% stake in its for-profit division, ostensibly for competitive, economic, or political reasons.
“One reason may be to buy political advantage against competitors. OpenAI faces competition from the likes of Anthropic, Google, Meta and xAI. Unlike its top rivals, OpenAI doesn’t have a large pool of cash or profits on its balance sheet to finance its data center build-out.
OpenAI plans to spend $600 billion on AI infrastructure by 2030, yet it is generating about $2 billion in revenue a month. An IPO would raise cash, but nowhere near enough to finance its ambitions. Perhaps Mr. Altman hopes that giving the government a stake will lower his company’s cost of borrowing.
Government ownership might also yield regulatory favors—such as faster approvals for models, permits for data centers or federal contracts. A government stake could provide an implicit backstop, making OpenAI too big to fail. The government won’t want to take a loss on its stake, even if it means keeping it alive like a zombie firm.”
To make matters even more complex, China itself had talks with leading AI companies about limiting foreign access to advanced AI models, both released and unreleased.
As it relates to compute, the entire world was supposed to be severely compute-constrained, and now SpaceXAI (lol) is selling compute to Anthropic, META is on record as considering a “cloud compute” business to sell excess supply, and AI giants are handing out tons of free computing power to grab startup share.
As noted earlier, despite having warned for years that AI could lead to a severe knowledge-worker catastrophe, rendering millions of white-collar workers obsolete, executives like Dario Amodei, Sam Altman, Mark Zuckerberg, and Andy Jassy are now walking back those claims. Writes the Wall Street Journal:
“Collectively, the narrative has shifted from worker-light doomsday scenarios caused by AI to a future in which workers keep their jobs—and get a productivity boost.”
Even mortgage-your-home-but-don’t-sell-Bitcoin Strategy CEO Michael Saylor is breaking his promise and “increasing his USD reserves” (read: selling $BTC).
In short, the world simply isn’t as straightforward as many of the armchair philosophers pontificating about the future of AI would like it to be, no matter how deeply they think about their predictive scenarios.
And none of this even begins to unpack the complexity of supply chains, manufacturing realities, raw material availability, geopolitics, and war.
What’s more, if society does decide that things are moving too quickly, and that AI is the scapegoat, there will be resistance. We’re seeing early signs of this manifesting already with Sam Altman’s home having been attacked, for example, and the first protests in front of the frontier labs’ offices being held this past weekend.
Unknown Unknowns
For a number of reasons, AI is a particularly difficult technology to understand deeply. It involves advanced mathematics, quantities of data that defy comprehension, and the industry’s leaders seem particularly ill-equipped to communicate anything to the general public.
It’s in part for this reason that these long narrative predictions are so popular: they’re sensationalist and engaging, but they’re very difficult to dissect and evaluate, let alone disprove. It should come as no surprise that the earliest examples of this genre were the most compelling, for we knew even less then than we do now about what AI would eventually be capable of and how it might impact humanity.
But the problem with these predictive exercises is the inherent entropy in the system and what Donald Rumsfeld called the “unknown unknowns.” As Rumsfeld wrote himself once he left office:
“The idea of known and unknown unknowns recognizes that the information those in positions of responsibility in government, as well as in other human endeavors, have at their disposal is almost always incomplete.”
There’s a selection bias buried in all this uncertainty, too. The scenario writers reach for the unknown to justify alarm (and action), but if we genuinely can’t see what is coming, it’s as likely to be a breakthrough as a catastrophe.
Redirection
Nobody can forecast with any confidence what superintelligence looks like or the consequences it might have on humanity. We can, however, forecast what the world will look like if energy becomes cheap and human lifespans stretch by a decade or two. In fact, intelligence’s real dividend lies in compounding engineering and design capabilities aimed at physical problems. But perhaps that’s too evident, too boring, or at odds with the authors’ intent (which we’ll get to below).
Energy, longevity, and robotics, for example, are all set to explode in the next decade, especially as we learn to harness the compounding effects of AI. The intelligence explosion, on the other hand, is hard to forecast because its defining feature, runaway recursive self-improvement, has no precedent.
Commonwealth Fusion Systems is finishing a compact nuclear fusion reactor that should reach net energy in 2027, with a first commercial plant under contract in Virginia and a billion-dollar power deal already signed. The FDA has cleared the first human trial of cellular reprogramming, a therapy that resets the age of cells; in mice, the same approach has extended remaining life by more than a hundred percent.
In robotics, intelligence finally has to touch matter, and the part being built now is the software that lets a machine move at all. Dimensional ships an open-source operating system that sits between an AI model and a robot, so an agent can drive a machine in plain language instead of the years of bespoke integration the field runs on today. The bet is that as robot models commoditize, the durable position forms in the software plugging them into the physical world, the same logic I have argued about the models themselves. (Disclosure: Dimensional is a Nazaré Ventures portfolio company.)
In short, apocalypse and superabundance are sideshows. Hypothetical narratives draw attention away from the real future (the tangible, near-term stuff actually being built), and direct it towards an imagined future (superintelligence, takeoff, the switch) despite the former being both more knowable and more important. The main event involves energy, longevity, and robotics rather than some self-directed robot demi-god with its own desires.
The Prophet Motive
All of this is sitting in plain sight, so why is the oxygen going to extinction instead of abundant energy, longevity, and robotics?
AI2027 was mostly legible as fiction. People referenced it to demonstrate “[AI alignment] deserves attention,” but almost never literally as “this exact sequence is happening.”
AI2040 differs from most of the “narrative prediction” genre in its prescription. Despite numerous “disclaimers,” they treat the underlying assumptions as settled, which is precisely what I articulated as a mistake in my last essay.
San Francisco and the AI community echo chamber now appear to be in an ever-more-grandiose competition to diagnose problems, prescribe solutions, and predict what will happen to the world when the low-probability-but-high-impact scenarios they obsess over finally come true.
The trouble is that they know their genre is particularly compelling at the moment. They’re manufacturing backlash and calling it a warning. Why, then, do such smart people use hyperbole like species-level extinction when they claim they want to be taken seriously?
They believe it literally. The “Don’t Look Up” scenario. Nobody who thought extinction was imminent writes a branching interactive website with five selectable endings and footnotes about their timeline updates.
They believe some real tail risk and knowingly inflate the vividness and the certainty. Classic “millenarian framing,” because calibrated uncertainty doesn’t compel policymakers to act, but “we all might die” certainly does (especially if the public believes it). This is motivated epistemics: identify the imagery that works, then backfill the conviction.
They don’t really believe the scenario and deploy it as an instrument. For relevance, influence, funding, power, organizational survival, regulatory capture, etc.
You don’t need to adjudicate which is actually true, because it’s at least one of them, and it might be a bit of all three. Whichever it is, extinction is a premise that renders the prescription unquestionable because it transcends “practical” considerations such as cost, enforcement, and other “messy middle” variables.
Either way, it’s difficult to disassociate the substance from the objective, which is clearly some form of direct or indirect influence over what happens in the future. There’s a particular arrogance in deciding, on behalf of everyone, what the future holds and who is fit to be trusted with it. At least representative government officials are elected. The scenario writers (along with labs and their leaders, like Anthropic and Dario Amodei) have appointed themselves as the ones who see clearly, while the rest of us blunder toward catastrophe. By consequence, they see fit to propose that the world should pursue their prescriptions because they’ve “seen the future first” and know how this ends.
Prediction-as-Value
In October of last year, Alex Danco published a brilliant piece on Prediction: the Successor to Postmodernism.
It argues that “prediction” is the cultural movement replacing postmodernism: the master frame for how we make meaning, build businesses, and find purpose in the AI century. It’s thoughtful, well-written, and helpful for understanding the cultural foundations that beget things like financial nihilism, memecoins, and prediction markets and, in turn, how they relate to artificial intelligence. Of particular importance is the following passage:
How early or late you are to something is now an essential component of your relationship to that thing. The timelines and reels that represent “what is going on” are increasingly about a single meta-topic: are you predicting it, or is it predicting you?
Many of the narratives being published from the belly of the beast appear to desperately want to be the former, but upon further inspection are often the latter. Offering thoughtful, structured arguments about where the world is going and how AI fits within it is like a rite of passage that the most enlightened in the industry must endure time and again to solidify their legitimacy.
But it’s gone just a bit too far. The echo-chamber could use less extreme philosophizing and a bit more levity and humility. Hotz is helpful here again.
“AI 2040 includes this picture of a datacenter in the ocean. Just like vaporware, you can generate a picture easily. But in reality, you have to deal with supply chains. You have to deal with them shipping you the wrong part, the thing not meeting the spec, it randomly failing after 20 minutes, the chip warping in the reflow oven. Did you consider the barnacles?”
