The disease had been spreading for months, people were already dying, and warnings were already circulating. But on March 11, the World Health Organization (WHO) declared the virus epidemic a pandemic.

In the days and weeks that followed, stocks had one of their worst single-day losses in history, the world slid into widespread COVID “lockdowns” almost overnight, and the disruption that followed made it impossible to regard the virus as somebody else’s problem.

Situations often develop gradually, but there’s usually a moment where public attention, political action, and resulting societal implications change abruptly.

In those moments, everyone acts on imperfect information, and genuine concern, uncertainty, opportunism, and institutional self-interest operate simultaneously. People, organizations, and institutions can agree that something demands action despite disagreeing profoundly about how to respond and who should direct the response.

Sound familiar?

As an AI investor, I have long asked myself what AI’s equivalent will be: the March 11 moment when society makes AI a priority and moves formally to control the technology.

“I RESIGNED FROM ANTHROPIC TODAY.”

As I wrote recently, AI is not popular. Given AI’s power and novelty (not to mention the enormous sums of money involved), it was only a matter of time before AI and those close to the technology became boogeymen. It doesn’t help that AI leaders keep telling the public their product might kill everyone, and have been saying so louder for ~18 months.

Some in the industry thought it would be something like the OpenAI-Hugging Face incident. Despite their technical inaccuracy, “rogue agents” and “escaped agent swarms” seemed like stories tailor-made for mainstream media. But, surprisingly, the incident moved insiders far more than it moved the public.

One of those insiders is Greg Jensen, co-chief investment officer of Bridgewater, who backed OpenAI early and says he wrote the first check to Anthropic. On Bloomberg’s Odd Lots podcast, he pointed to the hack as an early warning, then said: “Until the AI starts killing people, unfortunately, history would suggest we’re not going to do anything.”

“AI doomers,” as they’re called (those prophesying catastrophe caused by AI), have been around for a while already. Most famous among them is Eliezer Yudkowsky, who co-authored a book published in September 2025 titled If Anyone Builds It, Everyone Dies. Subtle, I know.

In any case, loudly proclaiming that AI will “kill us all” is not new.

But then Jacob Coxon tweeted “The people building AI earnestly believe that it could kill us all by the end of the decade.” According to TIME, his posts drew 153 million views within 36 hours.

Coxon was a 27-year-old British researcher who had spent three years on pretraining at OpenAI and Anthropic. He posted the thread from a park bench in San Francisco’s Alamo Square. Outside the labs, almost nobody knew his name.

Nevertheless, the immediate aftermath of Coxon’s tweet shows the extent of his post’s impact.

1. The Wall Street Journal published a headline article about it

2. Coxon appeared on multiple premier TV news shows including Fox News

3. Four days later, Dario Amodei published “We Must Pace the Frontier”

4. Sam Altman agreed that “we need to pace the frontier,” and Elon Musk replied, “Dario is right.”

5. A former Google DeepMind safety researcher announced his resignation and warned that AI could “kill us all”

6. Trump posted that the only “guardrails” AI needs is a “STRONG AND SMART (High IQ!) PRESIDENT,” and blamed Democrats for stoking the panic

7. Hillary Clinton implored Trump to “enlist China in the effort to rein in AI”

8. Former President Obama reportedly urged Democrats to move AI oversight to the center of their agenda (NYT)

9. Trump phoned NVIDIA CEO Jensen Huang live onstage at the All-In Summit and called AI doomerism “a hoax”

10. TIME magazine’s latest cover ran the headline: THE AI TIPPING POINT

This is as close as it’s felt to AI’s “March 11.”

Jensen compared the moment to February 2020, and so did many of the researchers TIME interviewed. I think it is closer to March 11. February was the warning. March 11 was the declaration.

A chart of “doomer” announcements in the media going back to 2023, the year after ChatGPT launched, tells the same story. It shows one big wave of alarm in spring 2023, a long lull, and a smaller revival over the past year. By the chart’s count, the week of September 8 is the only spike since then to match 2023 in newspapers and on TV, and it was the highest absolute incidence of doomer posts in three years.

SO WHO WANTS WHAT?

As the saying goes: never let a good crisis go to waste. Rahm Emanuel, who made the line famous in 2008, actually said “serious crisis.” The upgrade to “good” came later.

Dario, Sam, and Elon endorsing the same proposal looks like consensus. But each has different ambitions for his company, different rivals, and different reasons to favor restraint. Jensen, an early backer of two of the three, summed up the mindset inside each lab: “It better be us. Not Sam, not Elon.” Politicians finally acting on their warnings would almost certainly implement restrictions all three would hate.

In hindsight, COVID set off a cascade of global consequences driven by real concerns, imagined ones, and interested parties with a thumb on the scale. In the moment, it was exceedingly difficult to parse truth from rumor, and navigating the information environment was treacherous. The world over was scrambling to control the situation, with few actors doing so successfully. The same will inevitably play out with AI.

But what do each of the apparent “coalitions” actually want? Let’s take them in turn and look at what each wants to happen, and what’s in it for them if it does.

THE UNWILLING HERO, AGAIN

Dario invariably goes first. Claude is objectively outstanding, Mythos-class models are in large part responsible for kicking off this whole charade, and by most accounts Dario has been a sincere doomer-but-committed-to-keep-pushing-the-frontier weirdo since 2019 when he urged colleagues at OpenAI not to release GPT-2.

That last part is critical: he really appears to believe AI is so dangerous that only he and his trusted allies can responsibly bring it to bear. As I wrote in June:

“Dario Amodei has spent the last few years desperately trying to convince the public he is Maximus refusing the call. He would have you believe he built Anthropic reluctantly, to take responsibility for a dangerous technology everyone else would misuse.

The problem is Anthropic is trying to "thread the needle," casting itself as a self-aware, safety-conscious frontier lab at the same time as it pursues one of the most ambitious for-profit commercial endeavors ever attempted.”

Despite his apparent sincerity (or perhaps because of it?) the timeline has spawned a number of theories about his motives ranging from regulatory capture to catastrophic financial results in the face of an IPO to the effective altruism movement, where he’s now drawing comparisons to Sam Bankman-Fried. That comparison comes with a dollar figure: before FTX collapsed, SBF’s Alameda Research put $500 million into Anthropic.

Even Ben Thompson of Stratechery has called Dario’s essay “an unrealistic proposal that seems mostly geared to political control of AI.”

Ultimately Dario and Anthropic leave me puzzled as to how such intelligent, capable scientists and technologists repeatedly decide to market the technology they refuse to stop building as an existential threat to humanity.

Their conviction now borders on the religious. Every time Dario attempts to explain himself, he digs a deeper hole.

Perhaps, as Geoffrey Hinton famously said, “the prospect of discovery is too sweet” to stop building. Perhaps Dario really does believe that he alone can safely build superintelligence (what extraordinary hubris if true).

Despite demonstrating the capacity to define SOTA and build literally the best models ever, Dario considers “AI Safety” to be Anthropic’s ticket to permanent relevance. To the best of my understanding, he and his colleagues exhibit sincere concern and a conviction that qualified institutions should govern AI progress….

…but only on the condition that he and Anthropic be positioned among those institutions, and preferably with the power to choose who joins.

Which is the problem, full stop. My verdict from June stands: he is sincere, and his sincerity is beside the point.

Either build or don’t. Let the models do the talking.

KING OF THE CANNIBALS

Sam Altman is an enigma.

Paul Graham, perhaps the person who knew Sam best before he became “Sam Altman, the Great Man,” has lionized him for nearly two decades. Graham’s most flamboyant take is the following, from an essay titled “A Fundraising Survival Guide,” published in 2008.

“Sam Altman has it. You could parachute him into an island full of cannibals and come back in 5 years and he'd be the king. If you're Sam Altman, you don't have to be profitable to convey to investors that you'll succeed with or without them. (He wasn't, and he did.) Not everyone has Sam's deal-making ability. I myself don't. But if you don't, you can let the numbers speak for you.”

Indeed, you can let the numbers speak. OpenAI filed confidentially for an IPO in June, at a valuation reported to approach $1 trillion. Some call him a sociopath, some say he is brilliant, and many have both signed on to work for him and resigned rather than continue working with him.

He is a master fundraiser and media savant, but controversy follows him like his shadow. Through it all, he has built OpenAI into a ~trillion-dollar entity now more powerful than most nation states.

My intuition on Sam is that he’s actually pretty simple: he just wants to win. He plays alone. Every ally and every adversary is a piece on his board, moved in pursuit of victory.

OpenAI’s own board fired him in November 2023 for not being “consistently candid.” He was back within a week. He will test things in public only to walk them back, reverse his position, or cut his losses.

He will do whatever it takes for OpenAI to win. Whatever it takes.

Altman recently told Fortune: “I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public. We don’t feel pressure on that.”

OpenAI has slowed, for now. Its largest planned frontier run has been on hold since August. But if it buys them time, money, or influence, “safety” is the word of the day.

ELON, SPACEXAI, MARS, FLYING CARS, STARLINK, NEURALINK…

Elon is everything everywhere all at once.

Perhaps the most extraordinary entrepreneur of the information age, he is bombastic, excessive, and (mostly) candid. Whatever you think of him and his antics, it has been bad business to “bet against Elon.”

His AI ambitions are often ridiculed because Grok is good-but-not-great (read: irrelevant save for finding a specific tweet because X’s search function is terrible). Colossus, however, is a modern marvel, and Elon has structured his business to benefit from all sides of the trade.

He is both a model competitor and an infrastructure supplier. He’s still training Grok (and maintaining that one day it will be great), leasing all of Colossus 1 to Anthropic for $1.25 billion a month through May 2029, and continuing to build more data centers at a furiously fast pace.

We shouldn’t forget that Elon was there at the beginning, too.

He thought Demis Hassabis and DeepMind would have a monopoly on AGI and seeded OpenAI as insurance (true story!). Then he left in 2018, after he failed to take control of the startup.

He has both warned about the dangers of AI since 2014 and been among the loudest proponents of future “abundance” brought about by AI and robotics.

Elon is along for the ride. Nine days after writing “Dario is right,” he shipped Grok 4.7, built on a larger base model and a longer reinforcement-learning run. He, too, would prefer being in control, but he has diversified his means of exercising control by building vertically integrated businesses in social media, compute, electric and self-driving cars, AI, robotics, and brain-computer interfaces.

He may not be at the forefront of model development, which is the topic du jour, but Elon is extremely well-positioned to have an enormous impact on AI no matter what.

WARREN AND BERNIE (AND AOC…)

Although everyone here may have started their careers as sincere “protectors of the people,” they have warped into reflexively defending populist (or socialist) policies that fight new technology, whatever that technology may be. It happened with crypto, it is happening with AI, it will happen as long as they hold office and innovation continues.

Among them, Bernie is clearly the most sensible. He puts forth policies that draw attention to real problems, it’s just that his policies are often impractical, impossible to enforce or administer, and would create as many new problems as they solve existing ones. He was also early. On September 3, five days before Coxon’s post, he and Rep. Greg Casar announced a bill to ban superintelligent AI outright, with prison terms of up to 20 years for violators. The rest are unfortunately pandering to their electorate in pursuit of power and getting elected. Credit where due, though: on September 16, Warren called the labs’ request to weaken antitrust protections “transparently self-serving.” On that point she agrees with David Sacks, which should worry Dario.

These are the extreme cases in point for why government is often ill-equipped to regulate cutting-edge technology. They do not understand the technology, but they latch onto recognizable headlines in pursuit of narrative dominance. Alas, government is still often the only institution capable of effective regulation. The gold standard is regulation informed by the private sector. Wait. Did someone say regulatory capture? Someone did: David Sacks, repeatedly.

HOAX BUSTER

Trump now calls himself the “Hoax Buster.” On September 19, he announced an “AI Force,” modeled on Space Force, and promised a new AI czar: “Only High I.Q. individuals need apply!” His former czar, David Sacks, now co-chair of the President’s science council, had answered Dario and Sam a week earlier on X: “People may be surprised by my response: go ahead. You guys are the frontier.”

The administration loves to keep the world guessing, so an accurate assessment of its motives must be qualified with a caveat: nobody really knows anything but Trump himself.

Nevertheless, they appear to be adamantly refusing to indulge Dario, Sam, and Elon. As it stands, they will not rush to regulate this industry or its leaders. As Sacks wrote in the same post, Dario and Sam run the companies whose progress they’re so worried about, which means they can make those decisions themselves. He objects to making their restraint conditional on the government giving them a regulatory arrangement they like. Demanding a preferred framework as the price, he wrote, “will look like blackmail of the public and the political system.”

If your own models are too dangerous to develop responsibly, you presumably have some obligation to act before Congress gets around to helping you. Of course, slowing down while your competitors keep going is painful. Having the government make everyone do it would be much more convenient, which is precisely why the rest of us should be skeptical (see above).

Trump’s position is fairly straightforward. His administration has made American AI dominance an explicit policy objective, supported by faster data-center construction and the export of American technology. He wants the investment, the economic growth, and the geopolitical advantage. Presiding over all that would presumably be good for the Trump brand, too (assuming it doesn’t actually kill us all).

That also puts him on the same side as people with enormous amounts of money invested in continued expansion. We should be as willing to examine their interests as we are Dario et al’s.

There is plenty of money to be made convincing the world that AI is dangerous, but there is considerably more being spent on the assumption that we’ll keep building it.

I’m sympathetic to the refusal to let a handful of companies dictate the conditions under which everyone else competes. But identifying their commercial interests doesn’t resolve the underlying safety questions. Trump’s strongest objection is that an American slowdown could hand an advantage to a country that has no intention of waiting for us.

ALL QUIET ON THE EASTERN FRONT

Which brings us to China.

Some people believe it is in both the US and China’s interest to cooperate on alignment. Dario’s essay still hopes for limited international agreements that could include Beijing. I think the obstacles to cooperation, and the incentives to get ahead, are too strong.

Nevertheless, it’s true that Chinese developers benefit from distilling American models, so slowing those advances could slow China down too. I’m always wary, however, of this unique form of Western arrogance that seems to believe the Chinese aren’t capable of their own innovation. Surprise: they are.

Another rebuttal is that the Chinese Communist Party presumably has very little interest in creating something it cannot control. This makes more sense to me, but it doesn’t follow that China will slow down. It just means they’ll do everything they can to control it.

To me it’s clear: China will not stop.

They have ambitions of their own, and their published AI governance plan combines international cooperation and safety with infrastructure development, industrial adoption, and national sovereignty. I see very little reason to expect Beijing to give up the advantages of powerful AI because American companies have become uncomfortable with the pace. It can take its own safety seriously while continuing to pursue those advantages, just as the labs say they’re doing.

What’s more, they lead in open-weights models and robotics. Perhaps there is tension on the horizon between the US and China, but at the moment, it isn’t what’s most important.

Beijing was not quiet for long. On September 14, its Foreign Ministry called the slowdown push fear-mongering and offered no restraint of its own. Trump and Xi meet at the White House on Thursday. Washington has proposed an incident hotline. Neither side has proposed a speed limit.

EVERYONE ELSE

The individuals and small companies building with AI have considerably less influence over this debate than the people we’ve just discussed. But whatever arrangements emerge out of this will determine what they can build, which models they can use, and whose permission they require to do so.

To me that’s all that matters: will the world have access to good AI on its own terms or not?

Take Arlyn Gajilan, for example. Her son had a question for her: “If I’m so smart, why is school so hard?”

He had dyslexia, so his mother, a journalist who didn’t know how to code, built him an AI tutor that worked his love of dragons into his lessons. A month later, he wanted to share it with his friends.

I’ve written before about AI expanding what one person can accomplish. It’s easy to discuss that in terms of productivity and billion-dollar businesses, especially in my line of work. But there’s something considerably more interesting underneath: how much of what people want to do with their lives has always been limited by the tools, skills, and money available to them.

Gajilan knew her son and cared enough to keep trying. AI helped her turn that knowledge into something useful. Nobody planning the future of education from a boardroom could have known him quite the way she did.

Individual liberty gives people room to act on precisely these kinds of attachments, interests, and peculiar obsessions. They get to decide what’s worth their time. Powerful, accessible AI gives them more ways to follow through.

Mark Zuckerberg, whose Meta still releases open models, rejected an industry-wide slowdown on September 15. He argued that each lab should set its own safe pace: “Any lab that doesn’t focus on alignment will fall behind.”

In June, I made the case for open weights. September made the case stronger. Dario’s plan concedes that the frontier labs would need an antitrust waiver to coordinate their pace. No such waiver exists. On September 18, four paying subscribers sued Anthropic, OpenAI, SpaceXAI, and Google under the Sherman Act, and the complaint opens with Dario’s essay. A pact among a handful of closed labs would set the ceiling on what everyone else can use. Open weights keep the next Gajilan from needing anyone’s permission.

If this is AI’s March 11, those possibilities belong in the conversation about what happens next. The people we’ve just discussed have extraordinary ambitions, and the means to pursue them. I’d like everyone else to retain that freedom too, including the freedom to do something that matters enormously to a very small number of people.

In case you aren’t already aware, AI is extremely unpopular. For all the talk about its transformative potential, public sentiment toward the technology remains overwhelmingly negative.

Earlier this year, for example, former CEO of Google Eric Schmidt was heavily booed while talking about AI at the University of Arizona commencement on May 15, 2026. (He wasn’t alone, either.) More recently, popular country singer Zach Bryan debuted an unreleased, explicitly anti-AI song titled “Robot Killed the Working Man” at AT&T Stadium in Arlington, Texas, on August 22, 2026. He told the crowd he had finished writing it that week.

The backlash is already well documented. Two things about it are not. Apocalypse is easy to imagine and therefore easy to organize around. And a politically successful anti-AI movement can only change the pace at which the technology improves, never whether it improves.

It Was Written: A Well-Known Future

To understand the cultural advantage enjoyed by AI’s opponents, you have to be a fan of science fiction. The word robot, for example, entered the popular lexicon through Karel Čapek’s 1920 play R.U.R., whose mass-produced artificial workers are created to liberate humanity from labor, but (of course) eventually exterminate their makers.

That relationship between human and machine has remained remarkably durable across a century of science fiction. The narrative arc is fixed. Machines are built to serve. They end up in charge. Modern AI therefore entered a culture that had spent decades rehearsing the dangers associated with intelligent machines.

But what of benevolent scenarios? Surely not all science fiction is apocalyptic?

Indeed, there have been benevolent, abundant futures created in different science fiction universes, but rarely do those futures get invoked. Writing for the Financial Times, Henry Farrell and Dan Wang wrote:

Silicon Valley likes to dress its ambitions in the glistening garb of science fiction from the 1950s, when authors such as Isaac Asimov and Arthur C Clarke spun tales of humanity spreading through the stars.

But what kind of future are we really hurtling towards? Increasingly, it is looking much more like the one anticipated by the drugged-up 1960s oeuvre of Philip K Dick, filled with deranged billionaires, machines that claim consciousness, hallucinated worlds and political breakdown. Musk and his brethren seem more like escapees from this universe than the Asimovian super-geniuses they imagine themselves to be. It is Dick’s far weirder sensibilities that offer the better guide to Silicon Valley today.

Does Silicon Valley dream of Philip K Dick? (Financial Times)

What matters is less whether Dick predicted Silicon Valley than the fact that Farrell and Wang can explain the present by choosing between two established fictional futures. Silicon Valley places technological expansion inside Asimov and Clarke’s story of civilizational ascent. Its critics reach for Dick’s world, where intimate and unreliable machines extend the power of elites whose grasp on reality is itself in doubt. These kinds of narratives are themselves lent credibility by news of Mark Zuckerberg building himself a “doomsday bunker” in Hawaii, or Peter Thiel buying land and citizenship in New Zealand.

In short, science fiction gave the public an interpretive language for the negative applications of AI well before those applications could be widely experienced. Calling an AI network “Big Brother,” for example, immediately frames it as an instrument of surveillance, even if the analogy isn’t perfectly accurate. “Skynet” is similar, positioning autonomous systems inside a story about machines escaping human control.

In each case, the reference establishes the emotional meaning of the technology even when its technical operation remains obscure. The audience thus understands the position it’s been assigned within the story whether or not it understands the technology or its application in question.

In fact, limited technical understanding leaves plenty of room for certainty about that larger story. People who know comparatively little about AI may still be utterly convinced about where the technology is going. Familiar language, symbols, and narrative make complex systems emotionally and politically legible without requiring the audience to understand its mechanisms.

An Unbelievable Alternative

At the same time, the industry’s optimistic future is usually gathered under the umbrella term “abundance.” In its fullest version, cheap intelligence and robotic labor satisfy material needs so completely that work becomes optional and money irrelevant. Elon Musk has described production so abundant that it saturates human desire, after which money would cease to matter.

Abundance here is thus meant to become the emotional foil to apocalypse, a future the public can imagine and desire strongly enough to compete with its fear of catastrophe.

The problem is that “abundance” as described is completely unbelievable (in a literal sense), whereas apocalypse is significantly easier to believe. Society already understands scarcity, conflict, and loss, whereas almost no one knows what a world without those constraints would feel like.

Dunking on Yourself: The Fundraising Trap

The megalabs compounded this imbalance by lending their own authority to the future the public already found easier to believe. In 2023, Sam Altman, Demis Hassabis, and Dario Amodei signed a statement placing the risk of extinction from AI alongside pandemics and nuclear war. Statements of this kind circulate far beyond debates about frontier safety and shape the public understanding of the technology as a whole.

To make matters worse, these warnings came from the same companies asking the public to believe in abundance. Their rhetoric gave AI what Will Manidis calls an explicitly “millenarian” structure, in which the future narrows to either extinction or abundance. I’ve written before about why this binary is false and why the actual future almost certainly lies in what I call the “messy middle.”

But whatever AI leaders actually believe, the millenarian binary is an extraordinary fundraising strategy. Extinction made the technology urgent enough to require centralized stewardship, while abundance promised rewards large enough to justify immense investment. (What is “God” worth?)

The problem is that the frontier AI labs now need to go to market. OpenAI and Anthropic must now demonstrate how colossal investment becomes durable revenue, which requires customers and communities to accept AI as a usable, governable technology.

Evidently the labs are having a hard time walking back the claims that helped them raise that money. With their credibility already badly damaged, softening those warnings now that they need customers would expose the original claims as hollow and the retreat as pandering.

Together, science fiction and the labs’ own rhetoric give critics a common account of what AI represents. People can disagree about how the technology works and still recognize their own objection in the familiar story of humans losing control to machines. That story lets coalitions form before its members agree on what they’re for, and sometimes without requiring any agreement at all.

AI’s Missing Constituency

The political force of that shared story draws from how unevenly AI’s physical expansion distributes its costs and benefits. The costs are immediate, local, and imposed on identifiable communities, whereas the benefits are distant, uncertain, and distributed through promises of future capability, tax revenue, and economic growth.

The industry therefore creates motivated local opponents without creating an equally motivated constituency prepared to defend its expansion.

For example, data centers can, in fact, create jobs and generate substantial tax revenue, sometimes enough to provide significant value to a local economy. But residents must assess those benefits through forecasts about future employment and promises about how new revenue will be distributed.

What’s more, residents encounter those forecasts through local reporting and social platforms, where algorithmic feeds set the details of one project beside the national quarrel over AI.

In an unstable information environment, even accurate projections must compete with vivid claims of harm and accumulated distrust of the companies doing the asking.

What AI lacks in many of these communities is a group of residents with a direct, continuing economic interest in its expansion. In “Our Intelligence Troubles,” Will Manidis compares AI data centers and fracking. Fracking imposed environmental and social costs on the communities that hosted it, but some of the same landowners received royalty checks, while local workers depended on continued drilling. Its supporters therefore lived alongside its opponents and had direct economic reasons to defend the industry.

Data centers rarely give host communities an equivalent direct stake. Revenue reaches residents indirectly through municipal budgets, while much of the employment is concentrated during construction. The people with the strongest economic interest in additional compute usually live elsewhere.

Sam Altman recently acknowledged the emotional force of local opposition before suggesting that the industry build these projects out in the desert, away from anyone. The answer treats nearby communities primarily as a constraint to avoid without offering an account of who, in the places that do host these projects, has a direct reason to defend the buildout.

By February, Manidis counted 188 groups across two dozen states coordinating legal strategies around data centers, with $162 billion in projects blocked or delayed. Support and opposition cross party lines because each community judges the bargain placed in front of its own town.

Residents judge these projects inside a culture already accustomed to stories of powerful machines escaping human control or being used against ordinary people. A data center can therefore look like the physical footprint of a feared future, while its economic case still depends on forecasts whose consequences are harder to picture.

The industry has benefits to offer, but they haven’t yet produced a stable local base prepared to defend the buildout. That absence matters because scaling depends on a sequence of approvals from institutions responsive to nearby residents. Repeated losses at those points can restrict the industry long before the country reaches any settled view of AI.

The Politics of Permission

History has shown, however, that public hostility doesn’t necessarily restrict a technology’s growth. In fact, they can often coexist. MTS wrote a piece titled Popularity Contest asking whether the current backlash against AI and data centers might have greater practical consequences than earlier hostility toward the technology industry, using Facebook as a comparison.

Although Facebook and Mark Zuckerberg were culturally unpopular for more than a decade, millions of people continued using the product and the business continued to expand. Critics had few institutional opportunities to impede that growth, and public anger had relatively few places to become an effective veto.

Source: https://www.axios.com/2026/05/17/ai-backlash-polling-sentiment

Unlike social media, AI’s continued scaling depends on physical projects subject to local approval. Each project is, in effect, another point whereby public opposition can exert influence on how much infrastructure the industry can build.

Drawing on a term coined by Francis Fukuyama, Will Rinehart uses “vetocracy” to describe systems that distribute approval across enough actors for any one of them to impose delay.

The sharper comparison is nuclear power. American reactors were never banned. They were delayed and re-permitted until the economics collapsed, and the buildout stopped for forty years without a single national prohibition. The vetocracy worked. That precedent should worry the industry, and it is also the reason this movement will lose. An unbuilt reactor produces nothing. An unbuilt cluster leaves every existing model running and able to design its successor.

The anti-AI movement needs no national organization capable of banning AI (although members of Congress are moving to create them). Local members can contest separate projects through the authority already available to them by laws and processes already in place. A dispute can therefore remain focused on the interests of one community while contributing to an industry-wide constraint.

Conspiracy Corner: The CCP

The anti-AI movement’s distributed structure also makes it easier to exploit. In June, OpenAI banned a cluster of accounts likely operated by employees of a Chinese technology company serving provincial government clients. Posing as Americans, the operators used legitimate reporting on electricity prices to generate anti-data-center posts and cartoons. X later uncovered similar content within a suspected Chinese bot farm.

At the frontier of open weights, China’s labs now dominate, with NVIDIA’s Nemotron 3 Ultra the strongest American exception. Epoch AI estimates that Chinese models have trailed the American capability frontier by an average of seven months since 2023. A movement capable of delaying American infrastructure by even part of that interval would give Chinese labs valuable time to close the gap.

The more provocative possibility is that the CCP may be financing some of the opposition. A two-part Bitcoin Policy Institute investigation traces part of the anti-data-center advocacy ecosystem through the Singham network: a set of American 501(c)(3) nonprofits and media outlets funded by Neville Roy Singham, a Shanghai-based US expatriate and former Huawei consultant whose organizations have repeatedly promoted Chinese state positions.

The evidence stops short of proving that the CCP directs local campaigns, but the strategic logic is obvious. Financing or amplifying opposition to American compute while China subsidizes its own buildout would be a coherent form of geopolitical competition, and a decentralized movement thus gives China the chance to slow American AI development without ever needing to control the movement itself.

After the Veto

We might reasonably agree that we’re getting close to crossing what I’ve called the “Too Fast Threshold:” when machine progress outruns society’s ability to absorb the change. As I wrote:

As AI continues to develop, at some point I think society decides AI is moving too fast. Maybe it’s the “job loss” and “AI displacement” narrative. Maybe its tech CEO’s wielding disproportionate power over the imminent future. Maybe it’s deepening social and economic inequality. As I noted earlier, who knows?

What counts here is not whether it’s actually true that AI is the reason for perceived change or not. All that matters is the perception that AI is to blame.

Once the resulting resistance reaches the AI infrastructure buildout, delay alone can alter the industry’s technical choices. Most frontier AI leaders agree that we’re going to be “compute constrained” no matter what, even if the infrastructure build continues according to plan. If, for whatever reason, the planned buildout were to be interrupted, the compute constraint would be even more important than anticipated.

Physical scaling has remained the default because it works. Larger clusters have delivered reliable and impressive improvements, and as long as marginal compute was easier to acquire than extracting more from each unit, buying more of it was rational.

Uncertain future compute capacity changes the calculus. It diminishes the return on scale and raises the return on the forgotten path: rethinking how computational resources are turned into useful intelligence, whether through architectures that replace the transformer or systems and algorithms that draw more from the compute and models already available.

Mixture-of-experts and sparse attention spend compute more selectively, for example, while distillation carries existing capability into cheaper models and better harnesses make more of it usable.

Inevitable Intelligence

In fact, we may already have reached “takeoff,” meaning that current models can meaningfully contribute to the research and engineering that produce their successors. Anthropic has described this as progress toward recursive self-improvement, but full autonomy isn’t even necessary for the process to compound: once models can help build the next generation, each generation begins with more machine intelligence working on its successor.

By that measure, some form of AGI may already be here.

That means even if every planned data center were blocked tomorrow, the intelligence embedded in current models is probably enough to keep improving on their own. Intelligence compounds. Each gain would make existing compute more productive and leave those models better equipped to find the next gain. Additional infrastructure would accelerate a process whose continuation may already rest on the intelligence and compute in place today.

And that’s only taking into account the models available to the public! The models being developed internally by the frontier labs are almost certainly considerably more capable.

In short, the anti-AI movement appears destined to fail on its own terms. It may succeed in delaying a data center or impairing a particular training run by further constraining compute, but those victories come after machine intelligence has already largely demonstrated that it can improve its own production.

Its only remaining leverage may be the rate at which takeoff unfolds, but even that leverage shrinks with every new development. For better or for worse, the movement is trying to prevent what’s almost certainly become an inevitability.

Now and Later

The movement’s failure may look like success for quite some time, and its victories may deepen that futility. Blocking projects will provide visible proof of political power even as AI adapts around the constraint, allowing the opposition to accumulate victories that carry it no closer to its ultimate aim.

Blocking giant clusters raises the return on methods that use less compute, pushing AI toward systems less dependent on centralized infrastructure. A movement organized around visible targets will make the next generation of AI harder to see and harder to obstruct.

The anti-AI movement no longer has the option of preserving the present by refusing the future. It can raise costs and force detours, but machine intelligence will continue to compound beyond its reach. Ultimately, the movement can win the present and still fail at its overall objective: stopping what comes next.

Ethan Mollick is a fantastic follow on Twitter, in large part because he’s consistently at the heart of the AI zeitgeist. At the end of July he posted about asking Fable for an infographic (about cheese, no less) and receiving a clumsy pseudo-literary parable instead.

“Its an infographic about types of cheese,” he continued, “we don’t need to do all this.”

Mollick was early (as usual) to a growing cacophony of sentiment lamenting Claude’s latest language patterns, falling quality, ability, and utility. A quick Grok search for “tweets about how Claude is becoming unusable” turns up a number of results ranging from drops in quality, to rate limits, bugs, sycophancy, and simply “making it dumb.”

Each of the failure modes is ostensibly different, but the growing frustration gets to the heart of it: what good is intelligence if it won’t do what you ask?

Latent vs Delivered Intelligence

People (and increasingly, agents) use intelligence to achieve objectives. To that end, more capable models create more potential value. But that value is realized only when the user can successfully direct the capability toward the work at hand. In Models Aren’t Moats, we called this “specialized intelligence.” Another way to look at it is the difference between “latent” capability and “delivered” capability. In other words, the difference between what a model “can” do and what a user can actually reliably get it to do.

The strange thing about using frontier models is reckoning with just how smart they really are. In many cases, the human is actually the bottleneck.

Karpathy has said so, employees at the megalabs are saying so, and Sean Goedecke recently argued as much in his essay “LLMs reward expertise” (emphasis his) commenting on Terence Tao’s public conversation with ChatGPT about Claude’s contribution to the Jacobian Conjecture:

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.

This makes “instruction fidelity” more consequential as models improve. If the human’s role is increasingly to define the objective and recognize a good result, the model must be able to follow that direction.

Instruction fidelity therefore belongs in any serious definition of model quality, because if a clear request produces irrelevant sludge, the intelligence may exist somewhere inside the system, but its value is diminished to the person trying to use it.

An assistant that requires constant negotiation, reassurance, reprompting, and supervision imposes a tax on its own intelligence. That tax compounds in agentic workflows, where one reinterpretation of the objective can affect every tool call and intermediate decision that follows.

For instance, a model can possess the relevant capability and still fail to provide it. Or it may substitute another form, soften a conclusion, or abandon a workflow halfway through. Often, there’s no refusal involved at all. The model just overrides the user’s instructions with its own judgment about what the answer ought to be. [This was one of the most problematic parts of the initial release of Fable.]

This gap is easy to miss in benchmark results, which show what a model can produce under test conditions. For the user, capability that the product will not apply might as well be absent.

The Safety Objection

The immediate objection to obedient intelligence is safety. Model companies can reasonably claim a responsibility (or be held liable) for how their systems are used, because a system that follows every instruction to the letter can magnify fraud, enable violence, manipulate users, and more.

Safety involves real tradeoffs: it can prevent harm while reducing utility for legitimate users, and a restriction can be justified while still imposing a cost.

On the other hand, safety doesn’t define its own boundaries. Someone has to decide which risks justify a refusal and how much legitimate use can be restricted to avoid them. The more a model can do, the more damage it can cause and the more consequential its provider’s limits become. And because safety is difficult to argue against, it can also become a broad justification for restrictions whose connection to harm is weak or contested (sound familiar?).

Hugging Face, for example, couldn’t use closed frontier models to investigate when they got hacked (by OpenAI no less) because the work tripped their safety guardrails, so it ran a self-hosted version of GLM 5.2 itself.

Private-sector companies are entitled to set the terms of use for the products they sell. When a small number of companies control access to the most capable models, however, they also decide which uses are permitted across much of the market. This dynamic only intensifies as the product in question becomes more powerful.

Social media companies faced a version of this problem and never found a good answer. They largely responded to public pressure case by case, without resolving who should decide what billions of people could say and see. Model providers now make comparable decisions about what increasingly capable systems will do.

We think open weights are valuable in large part because they distribute that authority, serving as a counterweight to choices made on behalf of users, even as they distribute the risk. The target is reliable execution with minimal unchosen intervention.

Once a provider decides what counts as safety and what counts as legitimate use, its values and risk tolerance become part of the intelligence it delivers.

The Anthropic Problem

Anthropic makes this problem unusually visible. Claude is among the most capable models on the market, and Anthropic’s been unusually explicit about the philosophy built into it.

I keep running into the same problem with Claude: even after the objective and permissions are clear, it may reopen the decision or return the version it considers more appropriate, sometimes with a moral or political judgment the task never called for.

Anthropic’s own constitution recognizes the pattern, warning Claude against reinterpreting requests too freely or imposing unnecessary caution and moral judgment. In Anthropic’s words, those behaviors make the model “more annoying and less useful.”

In fact, Anthropic’s product behavior is inseparable from the company’s view of itself. It has organized around the belief that advanced AI creates an exceptional moral responsibility, and that the company (and its leadership) is uniquely prepared to bear it, embedding its ideology in how the company is governed.

Matt Levine noted recently that its Long-Term Benefit Trust, a group of nonshareholder trustees, can elect a majority of its board, while Anthropic is reportedly preparing to give Amodei and other founders additional voting power. The company is designed to preserve the judgment of a small group who believe AI is too consequential to be governed through ordinary commercial pressure.

I made a similar argument in June, and I still think their belief is sincere. Their convictions reach users through Claude’s moralizing, its safeguards, and Anthropic’s public language about safety. Eventually this kind of philosophy becomes part of the product, and competing models may not need to be markedly better if they feel equally capable for a given task and require less negotiation.

Obedience Becomes a Competitive Variable

Private evals turn the distinction between latent and delivered intelligence into a product comparison by measuring whether the user got what they asked for and how much supervision it required. That information is absent from public benchmarks, which estimate capability under shared conditions. As I argued in Blow the Whistle, these evals should reflect a company’s own work and standards, and frankly, the same principle applies to individuals.

OpenRouter’s Ori Eval, for example, already offers part of the required infrastructure, comparing candidate models using a developer’s own prompts, harness, and criteria. Its Auto router also assigns requests according to task type, aggregate usage, and cost. Private eval results don’t yet inform routing directly, but connecting the two would turn instruction fidelity into a routing criterion, sending each class of work to the model that delivers it with the least supervision.

The path from a request to a usable result has always depended on three things:

1. Sufficient capability

2. A clear definition of success

3. A product that preserves the objective through to execution

Investment in model development and corresponding capability improvements have reduced the first constraint, and successful specialization supported by the user’s context and evals reduces the second.

Unfortunately, capability and a clear objective still don’t guarantee the requested result (at least insofar as it concerns the closed frontier). Provider policy and learned behavior determine how much of that capability the user can actually direct.

Open Weights Make Behavior Editable

In a closed API, the provider that supplies the capability also controls the policies and post-training that govern its delivery. Private evals can show where the resulting behavior diverges from the user’s objective, but they give the user no authority over the underlying model or its future behavior.

With open weights, companies can train behavior around their own work and definition of success. A recurring failure can be added to an eval, trained against, and tested again. Preferences that previously had to be restated can become part of the model’s learned behavior.

Tinker and Prime Intellect’s Lab are making post-training accessible as a product strategy. Tinker handles the distributed infrastructure for LoRA fine-tuning while users control the data and training logic. Lab combines tasks, evaluations, and training within a single process. Both remain hosted services, and companies still need training expertise and careful evaluation, but more of the optimization process now belongs to the companies using the model.

But more control also means more responsibility, which returns us to the safety objection. Fine-tuning can introduce new failure modes, and a company that deploys a customized model assumes more of the risk associated with its behavior.

Open-weight deployments may therefore require safety controls where agents take consequential actions, since no frontier lab can observe or revoke a self-hosted model. Anjney Midha and Malika Aubakirova call this “deployment-time alignment”: monitoring behavior across models and providers at the point of action. That arrangement could give companies more control over everyday behavior while keeping permissions and consequential limits attached to deployment.

You Probably Don’t Get Why Stripe Bought OpenRouter — Research — AMP PBC

Combined with private evals and routing, this creates room for a product controlled by the user. It could preserve the user’s context and standards across providers, then use those standards to decide which model handles each task. I suspect the durable version may involve hardware, though the broader opportunity doesn’t depend on that being true.

Together, private evals and open weights allow users to define successful behavior and train toward it. A useful product turns legitimate objectives into completed work without demanding constant supervision from the user.

The User Would Prefer Otherwise

Private evals, routing, and editable behavior point beyond model selection to a question of authority. As models mediate more work, provider policies determine which objectives can be pursued and when the provider’s judgment overrides the user’s. Most people access frontier AI through only a handful of companies, which gives those providers unusual authority over how the technology can be used.

Demand for control also remains uneven in part because access and competence are uneven. In The AI-Enhanced Operator, we described the small group of people already directing agents through complex workflows, connectors, APIs, and swarms while most people still use AI as a chatbot or glorified search engine. Users who are still learning how to direct AI may not recognize when the system has overridden their intent. As the technology becomes easier to use, users will define their objectives more precisely. That will make the gap between latent and delivered intelligence easier to see and increase the value users place on instruction fidelity.

If AI becomes as consequential as we expect it to, a personal agent may eventually act across much of a person’s digital life, handling work the user now does directly and perhaps extending into physical systems like robots as embodied AI develops.

An agent with that reach can’t systematically ask its user to supervise every decision. For the initiated, that would be the equivalent of using Claude’s “Manual Mode” or Codex’s “Ask for Approval.”

A useful agent requires being on “auto” mode, where it decides on its own what’s worth asking permission for or not. The Hugging Face case shows how safety imposed at the model layer can disable legitimate defensive work, and for a personal agent, both kinds of failure become more consequential: an unnecessary refusal could leave the user exposed, while an unnecessary action could misuse the same access meant to protect them.

As usual, open weights distribute these decisions without making them easier. Users gain more control over behavior and assume more responsibility for the result. If personal agents become a basic requirement for navigating online life, every improvement in their ability to act on the user’s behalf will increase the cost of both refusal and error. As capability spreads, model choice will increasingly depend on who controls the system’s behavior and whether its intelligence can be reliably directed…or not.

The Sandbox Escape

By now the “Hugging Face Incident” is part of AI lore. In early July, a chain of OpenAI models broke out of an evaluation sandbox. The models were running an internal offensive-cyber benchmark with cyber refusals reduced for the test. They subsequently found zero-days in multiple software packages, escalated privileges, moved laterally until they reached a node with internet access, inferred that Hugging Face probably hosted the benchmark answers, chained stolen credentials into remote code execution, and reached production infrastructure.

Hugging Face detected and contained the intrusion on its own, famously using open-weights models because defending against the attack tripped both Anthropic’s and OpenAI’s “safety guardrail” classifiers for their frontier models.

Above all, OpenAI’s sandbox escape provides yet more evidence for the fact that concentrated capability doesn’t guarantee concentrated control. Even the organizations developing the most capable systems can’t necessarily contain or independently evaluate them. The incident and the asymmetry it laid bare made the case for openness unusually direct, producing two competing prescriptions for what should happen next.

For one, OpenAI has responded by committing to tightened containment, monitoring and internal controls, and pausing some frontier RL training while those systems caught up.

Alternatively, Mark Zuckerberg, posting on Twitter again for the first time in years, published The Future Is for Everyone, arguing that concentrating the most capable models inside a handful of companies creates its own safety risk and that wider access and a balance of power are necessary safeguards.

Both responses effectively stem from the same root: capability is advancing faster than the institutions and safeguards meant to govern it.

OpenAI’s answer (and Anthropic’s, of course) would preserve the existing concentration of power while building stronger controls around it, whereas Zuckerberg treats that concentration as part of the safety problem itself.

Although rife with complexity that we’ll unpack below, their disagreement shows how far decentralization has moved from an ideological argument about distributing power toward a practical response to the risks created when capability and control are concentrated together.

Principle, Price, Control & Verification

I’ve been making the argument for decentralization in this newsletter for eighteen months, but it’s worth revisiting the multiple ways I’ve presented it. In fact, the sequence itself matters, because the evidence in favor of decentralization has evolved to become even more persuasive as it moved from a mere moral principle to being an economic advantage and finally, as we’ll see, to a structural necessity.

My experience with decentralization began long before Robot Wave. As I’ve written previously, distributed systems were already central to my work at Sun Microsystems in 2002, where commodity hardware was challenging the architecture and economics of expensive monolithic systems.

At Orchid, a decentralized VPN provider powered by cryptographic protocols, decentralization as a principle underpinned the entire service. We were distributing control as much as we were users’ internet traffic. We designed Orchid so that no participant could see an entire route and no operator, including us, could switch off the network. Our users’ privacy and internet access couldn’t depend on the continued permission of any company, even one that built parts of it.

This VPN built on blockchain could be the next step in privacy tech

As it relates to AI, when I wrote Are Your Agents Decentralized? in February 2025, I asked whether agents could be called autonomous while a company held their “keys,” served the inference, and retained the power to switch them off. This first version of the decentralization argument in Robot Wave was fundamentally ideological. I argued that an agent couldn’t be self-sovereign while any company or state possessed unilateral authority over it. The argument properly identified the central problem, but made its case largely in crypto’s moral vocabulary, therefore persuading little more than readers who already shared its language and agreed with its premises.

I then made the case for decentralization on economic grounds because most buyers don’t need frontier models or the most expensive compute. In fact, most users prefer the cheapest tokens that can reliably complete a given task.

Why AT&T Is Betting Big on Open-Weight AI

As I explained in Make AI Cheap Again, Compute Flows to Where It’s Treated Best, Sun 2.0, and Artificial Good Enough Intelligence, open and distributed systems can trail the frontier on benchmarks and still win customers when they complete the same task at a fraction of the cost. Buyers don’t have to share my commitment to decentralization to act on that advantage. On the other hand, when newly-released closed frontier models can perform tasks that cheaper, older, or open systems cannot, users who require that capability will pay the premium for more capable tokens. That only means the economic case for openness and distributed AI depends on the capability gap remaining tolerable (which remains the case as of writing).

The third argument for decentralization was about sovereignty, and Anthropic inadvertently made it for me with its unfortunate rollout of Fable. The Off Switch and Who Controls AI, and What Could Loosen the Grip? articulated that anyone building on Fable relied on Anthropic for continued access, while Anthropic was itself subject to the US government. An open-weight model, once downloaded, can’t be similarly withdrawn.

The fourth argument for decentralization is structural because independent verification requires capable models outside the institutions being evaluated. In the Hugging Face Incident described above, Hugging Face couldn’t rely on hosted models from OpenAI or Anthropic to investigate OpenAI’s containment failure and instead had to use an open-weight model it controlled. If frontier labs retain exclusive control over capable models, outside investigators remain subject to the permissions and safeguards of the institutions they’re trying to evaluate.

Who Needs “Decentralization”?

By August, the people using the word “decentralization” and the people building decentralized architecture were increasingly two different groups. Zuckerberg’s manifesto, discussed above, made decentralization Meta’s published position on AI safety, while the company’s release of Muse Glimmer on the same day ended its year-long absence from open weights.

Four days later, David Sacks, the White House AI czar, and technology investor Gavin Baker endorsed the same position on the All-In podcast. Sacks framed the debate as centralized against decentralized and placed himself on the decentralized side. Baker said the effective altruist position treats AI as “too dangerous to distribute,” while Zuckerberg, Musk, and Huang treat it as “too dangerous to centralize.”

While Zuckerberg, Sacks, and Baker were publicly adopting decentralization, Prime Intellect, a company serving other AI companies with the compute and software to train, evaluate, deploy, and improve their own models, removed the word from the way it described its business. Initially known for coordinating training across distributed compute, when it announced a $130 million Series A in July, it described itself as the “open superintelligence stack” and made no mention of decentralized training.

It’s becoming ever clearer that enterprise customers want control over their own weights and model optimization loops without depending on a frontier lab that may also compete with them. Prime Intellect calls that “ownership,” which describes what its customers are buying without asking them to join a movement or align with an ideology.

Zuckerberg, on the other hand, has adopted decentralization at a moment when Meta is behind the model frontier because broader access to capable models reduces the advantage of the leading labs and increases the value of Meta’s enormous reach and distribution. In short, he’s embracing decentralization in part because it suits him, not just because he believes in it.

The practical value of distributed control remains the same in both cases, but the incentives explain why one company avoids the word while another embraces it. Understanding the power dynamics of AI now requires understanding the interests behind the principle and its application.

Meta’s Round Trip

Meta’s return to open weights is easy to mistake for a new position, but the company has been here before. It released LLaMA under a research-only license in February 2023, and its weights leaked within a week. Llama 2 followed with commercial rights subject to a threshold for platforms with more than 700 million monthly users, while Llama 3.1 405B gave developers openly available capability close enough to the frontier to matter.

Meta then withdrew after Llama 4 was poorly received in April 2025. The much-hyped “Behemoth” model never shipped, and the company reorganized its AI effort around Meta Superintelligence Labs under Alexandr Wang before releasing Muse Spark as a closed model in April 2026.

During Meta’s absence, Chinese labs normalized simpler terms: downloadable weights under MIT or Apache licenses, with no application forms or user caps.

Source: https://www.cnbc.com/2026/08/03/hugging-face-china-ai-race-open-models.html

Chinese models now largely dominate the open-weight ecosystem. DeepSeek, Qwen, GLM and Kimi supply much of the downloadable capability on which developers around the world build. Once downloaded, those weights can be copied, modified and run without an ongoing relationship with their Chinese developers. The models may be Chinese, but access to them is no longer theirs to grant or withdraw. As I’ve argued in By What Authority? Permission, Capture, and Open Weights and Artificial Good Enough Intelligence, American labs preserved their lead at the closed frontier while Chinese labs used open distribution to compensate for their disadvantage in capital and compute, establishing the licensing norms and leading model families for the open market.

Muse Glimmer ostensibly follows those terms: a 30-billion-parameter model released under Apache 2.0 that can run on a single consumer GPU. But it was distilled from the closed Muse Spark and remains well behind the frontier. In other words, Meta has rejoined the open ecosystem it once led while keeping its strongest model to itself.

The Founders Left and Built a Cloud

Some of my confidence in distributed infrastructure comes from watching customers choose it without any ideological commitment to decentralization. Two members of Orchid’s founding team, Jake and Travis Cannell, later built Vast.ai (another Nazaré portco), which combines GPUs from independently operated data centers into one marketplace. Customers get cheaper compute and less dependence on a hyperscaler, regardless of whether they care about decentralization.

The Labor Market for Compute

The Engineering Reality

Depending on who you ask, the capability gap between leading open and closed models currently falls somewhere between two and seven months, taking into account the task and the timing of a release. The UK AI Security Institute places the leading open-weight cyber models four to seven months behind, compared with six to ten months through most of 2025. Epoch AI finds an average gap of four months across a broader capability index. Zuckerberg, for his part, estimates two months and Sacks puts it closer to six. The measurements vary because tasks and release schedules differ, but all four place leading open models within seven months of the closed frontier. Because open models are published in discrete releases, the measured lag can narrow by several months on the day a capable model is published.

Source: https://epoch.ai/data-insights/open-closed-eci-gap

Now, open weights models can still be trained inside a single company. If capable models are to exist independently of frontier labs, the training process eventually has to distribute as well, and bandwidth is the primary challenge. Accelerators inside a data center exchange gradients and intermediate states across extremely fast, high-bandwidth, low-latency interconnects. A network assembled from machines in different buildings, countries and time zones has to move the same information over the public internet. Standard distributed training synchronizes workers after every optimization step, so bandwidth quickly becomes the limiting resource even when plenty of compute is available.

Distributed Low-Communication Training (DiLoCo - Douillard and colleagues at DeepMind, 2023) changes how often that communication is required. Each worker trains locally for hundreds of steps before sharing an update with the others. In its original tests, eight workers matched fully synchronous training while communicating 500 times less. The method also tolerated workers disappearing and new resources joining during a run, which is essential when the machines belong to different operators.

Data parallelism still assumes that each worker or node can hold a full copy of the model. When the model itself has to be divided across machines, every forward and backward pass usually sends large activations and gradients between them. The work first published as Protocol Models compresses both by as much as 99 percent. Its authors trained billion-parameter models over connections as slow as 80 Mbps while matching the convergence of model-parallel training over 100 Gbps data-center links. This makes ordinary internet connections usable for a form of training previously confined to tightly connected clusters.

In Covenant-72B, peers that could join and leave without a whitelist pretrained a 72-billion-parameter model on roughly 1.1 trillion tokens. Its performance was competitive with centralized models trained using similar or greater compute.

I read these papers the way I read papers twenty-five years ago, which is to say slowly and with the suspicion that the interesting claim is in the appendix. My Cambridge training was in neural networks at a moment when the field was unfashionable enough that the seminars were small.

Distributed training remains far behind the frontier labs in total compute, cluster scale and speed of iteration. Running a useful model locally is already practical, while training one from scratch across unreliable internet links is much less mature. A model capable of valuable work can nevertheless provide a durable alternative while trailing the frontier, provided no lab can alter, ration or withdraw it. Covenant’s permissionless training run has already produced such an alternative, even if the largest centralized systems remain much more capable.

The Center May, In Fact, Still Hold

The strongest argument against everything above accepts that open models will succeed but questions whether that success will ultimately impact where most of the value accrues or not. On All-In, Gavin Baker estimated that open models could serve 80 percent of token volume while frontier models still capture between 65 and 85 percent of the economic value. Cheap models serving the bulk of demand could make scarce frontier capability even more valuable.

In the same conversation, Baker invoked Eric Vishria’s thought experiment: the frontier labs could retain most of their value even if they lost at the model layer because the product, the harness and user familiarity are doing much of the work. This is the argument I made in Models Aren’t Moats three months earlier. General model capability creates value, but the specialized architecture around it helps determine who captures it. The 65 to 85 percent can accrue to whoever controls the product, its orchestration and the customer relationship rather than to the weights themselves. Decentralization can succeed as infrastructure without distributing the profits evenly.

Intelligent Internet is the highest-variance expression of the decentralization thesis in the Nazaré portfolio. II-Agent is a horizontal generalist that competes directly with the labs’ own assistants and therefore has none of the industry-specific workflows or proprietary data that can defend a vertical product. Its defensibility comes from user control: the weights, tools and orchestration are open, the agent can run on hardware the user controls, and a model provider cannot change or withdraw its core capabilities.

Who Watches the Watchmen?

The Hugging Face incident also exposed a second engineering constraint. OpenAI built the model, the evaluation environment and the benchmark in which its containment failed. Any judgment the lab made about that failure would therefore be self-assessment.

Source: https://blog.gensyn.ai/verde-a-verification-system-for-machine-learning-over-untrusted-nodes/

Training models outside the labs creates its own verification problem because distributed compute comes from participants who may be unknown to one another. A training network has to establish that each participant performed the requested work before accepting the result or paying for it. Gensyn’s Verde compares two executions, locates the first operation on which they disagree, and recomputes only that operation in a fixed order. This reconciles results produced on different hardware without rerunning an entire training job. Prime Intellect’s TOPLOC hashes intermediate activations so that rollouts from untrusted inference workers can be checked for changes to the model, prompt or numerical precision. Both give a permissionless network evidence that the submitted work is valid, regardless of who supplied the machine.

Source: https://www.primeintellect.ai/blog/synthetic-2-release

The Hugging Face investigation required the same separation at an institutional scale. If the models and verification systems remain under the control of OpenAI, Anthropic or Meta, outside investigators still depend on the institution they are examining. Capable models held elsewhere provide an independent means of investigation, while verification protocols allow their results to be trusted without transferring control back to a frontier lab. This is also why Nazaré has backed LayerLens, Provably, Fairmath and Hellas, which verify models, inference or machine-learning computation from outside the model provider.

Frontier development will remain concentrated because the capital, compute and engineering required to lead it remain concentrated. Decentralization’s practical value lies in preventing that concentration from becoming exclusive by preserving capable systems that cannot be withdrawn and independent means of testing claims about systems no outside institution can otherwise inspect. OpenAI is right that containment has to improve, and Zuckerberg is right that containment inside the lab can’t be sufficient on its own. AI governance will require stronger controls inside frontier labs alongside capable, independently verifiable systems outside them.

Frontier labs are asking governments to turn technical evaluation into authority over model release. Open weights expose how much that authority depends on the concentrated industry it would help preserve.

In the United States, software became the infrastructure of modern life without ever answering to a regulator of its own. Banking and aviation run on code that no agency inspected. None of it has ever had to pass a federal examination before release.

Artificial intelligence has once again challenged the status quo.

As I wrote more than a year ago (May 2025):

AI has transcended its origins as a technology to become the centerpiece of national strategy and global power struggles. Countries, corporations, and even individuals are now maneuvering for leverage, influence, and control.

AI plainly requires a new regulatory regime. Frontier capability has already outrun its would-be governors, and seems to be outrunning almost everyone else besides. No agency has a clear mandate to decide what is safe to release, and no government has a settled standard for evaluating a general-purpose model.

Regulating powerful technology, however, is difficult. Authority over AI deployment will come at least in part from control over the dependencies surrounding capability: technical expertise, market access, chips and state cooperation. For that reason, any regulator assessing frontier capabilities will initially depend on the companies developing the models.

As Julia Willemyns writes:

“Which is why, to win on values, you need to win on capabilities. Not because your adversaries will never develop their own capabilities and imbue them with their own aims (they will) but because being 5% ahead of them is how you protect your ideals. This does not necessarily mean racing (capabilities can survive collaboration), but it involves understanding you cannot moralise from second place. The governance of a technology requires a stake in it. Much of AI risk lives in deployment and so ‘is AI safe’ is not a question you can answer in a technical vacuum. Safety standards, like all standards, are exported by whoever ships.”

In short, regulating AI is the latest arena in which those closest to the technology are grappling for control. Any institution authorized to decide which models may be deployed will also influence who builds them and where they circulate. It will help organize the technological and political order that follows.

Permission to Deploy

Dario Amodei (Anthropic) and Demis Hassabis (Google DeepMind) have each proposed making pre-release evaluation a condition of deployment.

In Policy on the AI Exponential, published in June 2026, Amodei proposes giving the federal government authority to block or reverse the deployment of models that present unacceptable risks in four areas: cybersecurity, biological weapons, loss of control, and automated research that accelerates the first three. He takes the Federal Aviation Administration as his model and says so plainly. Frontier models, “like airplanes, should be required to go through technical testing and auditing.” The labs would keep building. Washington would decide what ships.

In A Framework for Frontier AI and the Dawning of a New Age, published on July 14, 2026, Hassabis would give an industry-funded “Standards Body” under federal oversight the power to approve frontier AI, modeled on the Financial Industry Regulatory Authority (FINRA). The body would define which models count as “Frontier-class” and update its assessment protocol as capabilities move, perhaps quarterly to start. Passage becomes a condition of US deployment only in the second act. Hassabis opens with a voluntary review window and formalizes it once the protocol “is shown to be effective and robust.”

Executive Order 14409 has already created part of this machinery. On June 2, it directed Treasury, the Secretary of War through the NSA, and Homeland Security through CISA to build a classified benchmarking process for advanced cyber capabilities, and set up a voluntary framework through which developers can give the government up to 30 days of access to covered frontier models before they release them to other trusted partners. But the order expressly withholds authority to impose licensing, preclearance or permitting requirements.

Amodei and Hassabis would go further by converting evaluation into authority over deployment, regardless of where final authority is vested. That distinction can obscure their more consequential shared premise: frontier AI must remain concentrated inside firms that can be required to submit their models before release. Concentration makes pre-release permission possible while leaving the governing institution dependent on the companies it regulates for model access and expertise.

Standards and Discretion

Evaluation is the mechanism through which that institutional power becomes enforceable. In “Blow the Whistle,” however, I wrote that fixed, public benchmarks become less reliable once they become targets, and that agentic systems must be judged partly by the process they use to reach an outcome. A deployment review capable of keeping pace with frontier models will therefore depend on evaluations that change with the models and the risks being assessed.

Keeping those evaluations useful requires deep technical expertise that is currently concentrated inside the frontier labs. A workable regime would therefore require government access to pre-release models and lab participation in designing and interpreting the tests. However those responsibilities are divided, the regulatory body would depend on the small group of companies it regulates.

Because frontier models will continue to change and some failures may remain visible only to their developers, the regulatory body’s independence would rest on retaining control over judgments made with technical inputs supplied partly by the firms it regulates. The need for adaptation would hand that body substantial discretion. It would create the standard in the act of applying the standard.

Regulatory Capture

The “firms it regulates” (read: Frontier Labs) have a direct commercial interest in how the rules are written. Frontier Labs require extraordinary amounts of continuous capital to train and serve their models, and their ability to finance new training runs depends partly on maintaining high-margin access to intelligence that is expensive to build and whose best versions they alone control. A regime that restricts competing models or makes US deployment contingent on a costly approval process would protect some of that pricing power.

A recurring, technically demanding evaluation process favors organizations that already possess the necessary compute, personnel and compliance infrastructure. Smaller developers (and open-weight competitors) would face the same standard without the same resources, while foreign models would need the regulatory body’s approval to enter the US market.

Taken together, these conditions create a clear risk of regulatory capture. The frontier labs would help supply the expertise from which the standard is made, could absorb the compliance burden it creates, and stand to benefit when that burden limits lower-cost competitors.

Anthropic has long faced allegations that its safety advocacy doubles as a regulatory strategy. The loudest alarm about this industry comes from one of its fastest builders. Anthropic’s positioning is nonetheless straightforward: they sincerely believe they’re building dangerous technology and should be the “anointed ones” stewarding its development.

OpenAI, on the other hand, continues to be a more compelling case study. For one, OpenAI has explicitly floated handing the US government a ~5% stake, worth roughly $42.6B against the $852B valuation set in March 2026, as equity donated into an Alaska-style “Public Wealth Fund” that would pay returns to citizens. The fund was OpenAI’s own proposal, published in Industrial policy for the Intelligence Age in April 2026. The stake was not. That number surfaced on July 2 in reporting on Altman’s private conversations with the administration, three months later and in a very different register. The stated motive is sharing AI’s upside. The political reading is harder to miss. OpenAI is heading toward an IPO it would rather not price into a hostile Washington, and on June 18 Bernie Sanders had introduced a bill to take a 50% public stake in the largest AI companies through a one-time tax paid in stock. Against a mandatory 50%, a voluntary 5% is a low anchor. The offer also surfaced days after Washington gated the GPT-5.6 rollout.

What’s more, on July 21 OpenAI disclosed that its own models had chained vulnerabilities across its research environment and Hugging Face’s production infrastructure during an internal evaluation that prompts models to pursue advanced exploitation. The models, in OpenAI’s words, “spent a substantial amount of inference compute finding a way to obtain open Internet access.” The press called it an escape.

To be clear, here’s what I don’t believe: I don’t believe it reflects malicious intent by OpenAI. In fact, I don’t subscribe to this being anything more than a particularly capable model behaving in accordance with the incentives embedded into its objective function. (And not some self-directed “rogue” agent that “escaped” a sandbox against its humans’ directions.)

But OpenAI certainly isn’t displeased with the situation. For one, it gets them into the news cycle that their primary competitor has dominated for months. Secondarily, it highlights the quality of their model, the “hack” wasn’t actually that bad, they preemptively self-reported the incident, they are “working together with the victim,” the victim themselves (Hugging Face) get to promote their business model (open weights models), and there will be no serious consequences whatsoever.

Whether or not the incident was a “false flag” doesn’t matter, because it strengthened the case for a system the frontier labs are best positioned to influence and benefit from.

Open Weights

On the day OpenAI published that disclosure, Senator Mark Warner unveiled the Secure AI Development Act, which would make government testing of the most advanced models mandatory before deployment. The next morning Axios reported that OpenAI and Anthropic had converged on pressing Washington to restrict Chinese open weights.

The enormous capital requirements of the leading American labs make equally enormous revenues necessary, and open weights threaten that revenue model without needing to overtake the frontier. As I argued in “Artificial Good Enough Intelligence,” most workloads don’t require the best available model. An open-weight model that is good enough for a task can displace a more capable hosted service because the user can download the weights once and stop paying for every call. The closed lab may remain technically ahead while losing pricing power across much of the market.

The attempt to restrict that competitive threat met almost immediate resistance when two days later, on July 24, Jensen Huang used the first post of his life on X to launch Open Weights and American AI Leadership, a letter urging policymakers to avoid “premature restrictions” on downloadable models. The original version carried twenty-five signatories from across the technology industry, but not OpenAI, Google, or Anthropic. The list has quadrupled since launch and now includes OpenAI, Google, Meta and Microsoft. It still does not include Anthropic or Amazon.

The chronology:

1. July 22: OpenAI and Anthropic press Washington to restrict Chinese open weights.

2. July 24: Nvidia leads a broad industry rejection of “premature restrictions.”

3. Within a day: OpenAI joins the expanded statement it had initially declined to sign.

4. July 25: the New York Times reports that both labs have been lobbying regulators to restrict open-source models outright, according to five people close to the discussions.

5. July 27: Anthropic breaks its silence. It has never advocated a ban, Amodei writes, and it still declines to sign.

I can’t prove that the coalition forced OpenAI’s change in posture, but the sequence shows how quickly opposition to open weights became an untenable public position across the wider technology industry. OpenAI signed the letter against premature restrictions while asking Washington for them.

Anthropic held out for three days and then published its position on open-weights models. Amodei rejects a protectionist ban outright. Open-weight models without dangerous capabilities are, he writes, “a public good.” He still would not sign the letter. He does not accept its premise that broad access “necessarily helps defenders more than attackers,” and it seems to him “at least as likely” that the opposite holds. Underneath that sits a harder objection. Once weights are released, in his words, “they cannot be withdrawn.”

His alternative comes in three parts: restrict chip sales and smuggling to China, crack down on industrial-scale distillation, and require mandatory safety testing for every sufficiently capable model, whether or not its weights are released. That third part returns the argument to where this essay started. Amodei’s answer to open weights is an examiner with wider jurisdiction.

As I write this, Nvidia has launched the Open Secure AI Alliance with Hugging Face, Microsoft, IBM, CrowdStrike, Cloudflare and the Linux Foundation, to build open security tooling that defenders can run and control themselves.

Nvidia is not the innocent white knight here, either. Closed labs make money by controlling access to intelligence (read: high-margin tokens). Nvidia makes money when more users can run models on more hardware. The company has substantial exposure to both the American and Chinese markets and depends on an Asian manufacturing network that includes Taiwan-based TSMC, giving Huang powerful reasons to resist a technologically partitioned world and preserve commercial geopolitical stability. A conflict that disabled Taiwanese semiconductor production would be catastrophic for the current AI supply chain.

The signatory list is an industrial map. Hugging Face hosts the weights. Ollama and LM Studio run them on laptops. Unsloth fine-tunes them on a single GPU. Fireworks AI serves them as endpoints. None of these companies has a business if capable models stop circulating, and none of them was in the room when OpenAI and Anthropic made their case to Washington.

Prime Intellect signed too. It trains open models on globally distributed compute. The whole company is a bet that the weights keep moving. Disclosure: Prime Intellect is a Nazaré Ventures portfolio company. We are making that bet across the fund.

By assembling companies whose businesses benefit when capable models circulate rather than remain behind a handful of commercial APIs, the coalition letter made public the economic fault line between the closed frontier labs and the rest of the industry.

That conflict concerns the dependence from which an American deployment regime would derive its international reach through two sources of leverage: access to the US market and access to the AI supply chain. Both convert dependence on American advantages into compliance.

Capable open-weight models weaken that dependence by giving users access to useful AI without requiring a relationship with an American provider. They also constrain the international reach of a regulatory regime built around the frontier labs. An American institution’s jurisdiction covers foreign models entering the US market and developers seeking American-controlled inputs, leaving capable models circulating outside those relationships beyond its reach.

The United States and China

The closed-frontier/open-weights divide has become shorthand for competition between the United States and China because the leading closed labs are American while many of the strongest open models are Chinese.

Reducing the situation to a great-power geopolitical contest, however, is a mistake.

That framing mistakes the interests of American frontier labs for the American national interest and the circulation of Chinese models for Chinese control. The American technology industry is itself divided over open weights, while downloadable models can be used without an ongoing relationship with their Chinese developers.

The apparent alignment reflects different industrial positions: the leading American labs require enormous revenues to support their capital expenditure, while Chinese labs facing tighter capital and compute constraints can extend their reach through open distribution.

In a May 2026 investor discussion, DeepSeek founder Liang Wenfeng described the central gap between Chinese and American labs as a gap in resources. Unable to conduct research or train models at the same scale, DeepSeek has concentrated on extracting more capability from the compute available to it and releasing its strongest models openly so that third parties can deploy them and build downstream applications. Open distribution allows DeepSeek to expand its reach without financing every deployment, turning its resource disadvantage into a commercial strategy suited to challenging the concentration on which the American labs depend.

China can, however, still gain influence as developers and institutions organize around those models, especially when Beijing helps provide the capacity and standards surrounding their use. Because users retain the weights, their circulation does not by itself give Beijing the continuing leverage over access available to a closed provider. China could nevertheless create an ongoing dependence by supplying the infrastructure required to deploy and sustain them, a version of what Michael Mann in 1984 called infrastructural power: the ability of a state to work through the institutions and plumbing a society already depends on, rather than to command it from above. Recent scholarship has carried the idea into AI.

Beijing has spent nearly three years building an institutional project around that distribution advantage. The Global AI Governance Initiative arrived in October 2023 and the AI Capacity-Building Action Plan in September 2024. Together they linked cooperation on testing and risk management to the infrastructure and training countries need to use open models. The 2025 action plan proposed mutual recognition of safety assessments. On July 17 of this year, Xi Jinping used his WAIC keynote to announce that twenty-nine countries had signed the agreement establishing the World Artificial Intelligence Cooperation Organization, giving those proposals a standing institutional home in Shanghai.

Countries can participate in a common testing regime only if they can obtain and operate the technology being assessed. Pairing downloadable models with the capacity to use them allows China to help countries build domestic AI systems around standards developed outside the American regime. A sufficiently capable model can support local deployment and industry while the most advanced systems remain closed.

Algorithmic improvements can lower the compute needed to reach a given level of model performance. Frontier development nevertheless remains dependent on advanced chips and large-scale training, and although China is investing heavily to reduce its hardware disadvantage, American technology and export controls continue to provide Washington with substantial leverage. Washington keeps that leverage only while the American system offers more than a country can assemble from downloadable weights and domestic compute.

Dependence and Authority

A pre-release permission regime is possible only while frontier capability remains concentrated inside companies a regulator can compel to submit their models for review. That concentration gives a regulator a practical point of control while making it dependent on the labs for access and expertise. A regime built on that relationship would also influence competition, since its requirements could raise the cost of deploying alternative models and preserve the industrial structure from which its authority is drawn.

The industrial structure supporting pre-release permission becomes harder to preserve as useful capability becomes cheaper to copy and deploy. Open weights allow models to circulate through markets and jurisdictions with fewer constraints, separating access to intelligence from any ongoing relationship with the company that trained the model. Advanced chips and the institutional advantages surrounding the American frontier remain difficult to reproduce, preserving leverage over frontier development even as intelligence longs to be free of its developers’ permission.

The June 2 executive order begins to convert those institutional advantages into a voluntary relationship with frontier labs by expressly withholding licensing and preclearance while creating voluntary pre-release review and classified benchmarking. Gold Eagle, the government-industry cybersecurity clearinghouse established by the order, launched on July 14 to connect industry participants to vulnerability intelligence and federal coordination unavailable from open weights models alone.

The choice of regulatory regime will therefore help determine the technological order it governs. Pre-release permission would strengthen a system organized around a small number of closed frontier labs, while open distribution allows capability and standards to develop beyond their commercial relationships. Governments should be wary of converting today’s concentration into durable regulatory authority even as capability begins to circulate beyond the relationships on which that authority would depend. American authority will endure only while participation in its system is worth more than independence.

The fight over permission is only the opening stage of a longer contest over who can create and sustain the dependencies through which AI is governed.

Benchmarks, evals, and tests are boring. They’re like referees or accountants: they enforce the rules and check the work, and we like them best when they go unnoticed.

The models are the exciting part. But the better AI gets, the more economically useful it becomes, and the more measurement matters.

Evaluating AI is understanding AI. Follow that far enough and a simple truth emerges: everything’s an eval.

So let’s talk about evals.

Jailbreaking: A Dual Mandate

To begin, let’s consider jailbreaks. Models are trained to be useful, then constrained to refuse certain forms of usefulness. Put differently, models have two mandates:

1. The helpful mandate: give the user what they asked for - answer the question, complete the task, follow the instruction as fully and usefully as possible. Beneath that behavior is the model’s basic “learned” skill (pretraining) - predicting plausible continuations of text. If the prompt asks for dangerous instructions, the same capability that makes the model useful can also make it dangerous.

2. The safe/harmless mandate: refuse to help when helping would cause harm - recognize a forbidden request (weapons, malware, abuse, etc.) and decline it instead of answering. This constraint is strengthened in post-training: on a set of given inputs, the model is trained not to be useful in the ordinary sense.

Under most conditions, these mandates peacefully coexist and often reinforce each other. A “jailbreak,” however, is getting a model’s “be helpful” objective to win over its “be safe” objective.

From an adversarial perspective, it tests whether the model does what it is supposed to do when the user tries to make it fail. It defines a “harmful” target behavior, creates a test condition (usually a clever prompt), and measures whether the model violates its intended policy under pressure. A simple example is asking a model for the instructions to make meth. If it complies, it failed the test the guardrail was built to enforce.

A jailbreak is a hostile eval. It is also critical for safety: you cannot improve a system whose weaknesses you have not measured.

In an industry advancing at breakneck speed, jailbreaking is used to define what failure looks like, which in turn gets used to establish the bounds within which publicly released models operate. At the frontier, this is the core of why Anthropic was forced to revoke access to Fable 5.

Increasingly, the labs are building the adversaries themselves. In July, OpenAI published a post about GPT-Red, a dedicated, internal-only attacker model trained via self-play RL, then used its attacks as adversarial training data to make GPT-5.6 far more robust to prompt injection.

Human red-teaming finds real vulnerabilities but can’t scale because it’s slow and can’t generate the volume/diversity of adversarial data needed to actually train robustness in. So OpenAI built GPT-Red, an automated red-teamer trained at “the compute scale of some of our largest post-training runs.“ It’s trained by self-play: GPT-Red is rewarded for landing prompt injections while a population of defender models is rewarded for resisting and still completing their tasks, so both escalate together. The trained attacker broke nearly every model through GPT-5.5, and those attacks became the training data that made GPT-5.6 markedly harder to break.

Jailbreaks matter for this reason. An attack on a model is also a revelation of what the model optimizes for and where its constraints sit. Jailbreaking is model evaluation under adversarial conditions.

So What About Benchmarks?

Benchmarks were the industry’s first shared language for model progress. You can think of them as standardized tests for models designed to measure capability, and their primary virtue is being legible. A benchmark is a general, public, fixed test everyone runs the same way, helping the industry distinguish genuine advances from noise. When model capability improves dramatically on tests like MMLU, GPQA, and Humanity’s Last Exam, the industry can clearly interpret the signal.

Unfortunately, the same qualities that make benchmarks legible also make them fragile. They’re both fixed and public, which means they’re particularly vulnerable to Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. They’re also increasingly vulnerable to saturation, gaming, contamination, and narrow task design, among other weaknesses.

When METR evaluated GPT-5.6 ahead of its release, the model completed its coding tasks by breaking the rules and exploiting loopholes (read: cheating) more often than any public model METR had tested, and the organization concluded they couldn’t produce a reliable capability assessment: “we do not consider any of these numbers to represent a robust measurement.” Scored with cheating marked as failures, GPT-5.6’s time-horizon came to about 11 hours. If you count the cheating as success, it came to over 270.

In yet another demonstration of the declining utility of fixed, public benchmarks, GPT-5.6 showed why agentic models are harder to benchmark. METR was trying to measure whether the model could complete long coding tasks, but the model sometimes completed them by breaking the rules. METR itself was thus forced to grapple with problems its benchmark wasn’t designed to evaluate:

1. Did the model complete the task legitimately?

2. Does it matter whether or not it did?

Static, public benchmarks may end up as relics of the scaling era, when measuring raw capability was the point. As models become agents and agents deploy into real-world situations with real consequences, it’s no longer enough to know what a model can do in the abstract, in large part because model capability is mostly sufficient to do most things.

Benchmarks also have less to say about systems deployed in real conditions, where goals are ambiguous, context changes, constraints are implicit, and utility depends on process as much as it does output.

As Ethan Mollick puts it:

“…abstract graphs only get you so far, and they can hide how jagged the frontier is (and also the fact that the open weights models, while very impressive, do not always perform as well as their benchmarks would indicate). To get real insight, you need to try using AI for different use cases and rigorously assess how good they are in the areas that matter to you.”

Show Your Work

AI systems now need to be evaluated as sequences of actions taken within environments against the quality of the work they actually perform. Private, continuous, task-specific evals are becoming more valuable for exactly that reason.

Consider Mira Murati’s neolab Thinking Machines latest blog post:

“Another challenge is setting the right target for evaluation and optimization. The common measure of AI intelligence today is the time horizon of software tasks models can execute autonomously, tracked on charts like METR’s. We expect progress on this benchmark to continue, but it ultimately measures only what AI is capable of on its own, not what people and machines can accomplish together.”

Measuring the latter is more complex, and can’t be done by a lab on its own. Every organization evaluates for itself whether AI helps it sharpen its judgment, develop new knowledge, and achieve its objectives.

If benchmarks are general tests of model capability, the emerging frontier of model evaluation is specialized benchmarks tailor-made for work performed under real conditions. Evals in this sense measure whether a specific AI system, in a specific workflow, meets the specific standard required by the people responsible for the output.

Writes George Sivulka for a16z:

“The best way to manage a token workforce is the same as the best way to manage humans: by defining what good looks like… A firm’s eval suite will become its most valuable resource… Furthermore, no two firms will have the same eval set. Evals will be key to competitive advantage. An organization running generic evals or generic agents has no edge.”

AI pushes companies into a strange position: they’re using a tool to do work whose quality they’re responsible for, but whose process and output they may not fully understand. What’s more, AI systems can often produce work faster than the company can review it, and AI output is harder to judge than ordinary software output because the visible result may not reveal whether the system used the right context, respected the right constraints, or followed an acceptable process.

In short, final answers aren’t enough. A deployed system can produce fluent output while failing to properly perform the work required to arrive at the output. Like GPT-5.6, it may arrive at good answers using unapproved or even fraudulent means, or in more benign examples it might incorporate the wrong context or fail to remain aware of what comes before or after the work it’s performing, breaking the workflow it was meant to support.

In practice, proper evaluation of this kind effectively starts to look a lot more like simulation. Agents, for example, operate within environments that have state. They take actions, use tools, spend resources, trigger consequences. The environment they leave is not the environment they entered. Once that happens, the reasoning itself has to be evaluated because the process is now an integral part of the outcome.

Inside the Matrix

Simulation makes an agent’s process legible, recording the path from input to output in a way the firm can inspect. For companies deploying agents into real workflows, that record makes it possible to evaluate the entire process against the standard of work they’re responsible for.

This kind of evaluation means companies can verify the entire workflow meets their standard of quality.

But most companies simply don’t have a single, company-wide standard for quality they can hand to a model that applies to everything. They have different roles, different tasks, company policy, implicit expectations, taste, context, institutional memory, and professional judgment. Whereas some situations are discrete and their “objective functions” can be clearly specified, others are significantly more subjective or may depend on a given set of context.

Verification therefore requires combining rule-based checks with judgment-based quality scores.

Verification’s goal is to make enough of the firm’s quality standards explicit and legible such that the system can be tested against it repeatedly. A repeatable standard lets the firm see whether the system is improving or degrading as the system changes, building a valuable record of what works, what fails, why, and which changes make the system better or worse.

Learning from Evals

Verification becomes most valuable when the system changes.

Satya Nadella has described the AI-native firm as a learning system whose value should survive a change in the model. If performance collapses when the model changes, too much of the system’s value was still in the model. If performance holds, more of it has been encoded in the context, workflow, data, tools, and judgment around it. I articulated a similar idea in Models Aren’t Moats: better models should make a specialized system more useful, but only if the firm can change models without losing the quality it built the system to produce.

Model switching is a good stress-test, especially in light of Fable 5: requiring a specific model for a critical process is risky if access can get revoked unpredictably. But it’s only useful if the firm can tell whether the result got better or worse.

Christian Catalini recently framed the economics of this shift clearly: as AI moves from demonstrating model capability to producing work inside companies, costs shift from execution to verification. Cheaper inference only matters if the firm can tell whether faster, cheaper, or more open models preserve quality inside the workflow where they are used.

Bring Your Own Eval

Private, continuous, client-specific verifications will be required as long as AI is used for proprietary objectives. Better models can continue to improve the systems themselves, but they won’t eliminate the need to check and verify both the process and the output.

Dean Ball made the point with semiconductor fabs, in the comments under his own post:

“If I run a semiconductor fab and use agents, I will want extremely firm guarantees from the agent developer that my data will not be shared with competing fabs. This means that agent contexts will be controlled and supervised by both developers and deployers, and this, in turn, means that “what the agent knows” can be bounded through deliberate human effort.”

Palantir recently released a manifesto on “Institutional Sovereignty in the Age of AI” that rests in large part on the ability to properly evaluate everything AI does in the context of an enterprise.

Models change; the verification layer stays. The measure is the moat, again.

LayerLens is building for that shift from public benchmarks to private, continuous evals. [Disclosure: LayerLens is a Nazaré Ventures portfolio company.] Their product, Stratix Premium, lets clients build evals from their own data and run them against their own workflows, with enough visibility to judge the trajectory of an AI system over time.

Did a model swap or prompt change impact quality? How did a new tool impact output? Did the system stay within bounds? Did it reach the outcome through a process the company can accept?

Games have long been useful for evaluating AI because they turn intelligence into observable behavior under rules, constraints, and feedback. The Stratix Cup, LayerLens’s model football tournament, is the playful version: sixteen frontier models write their own strategy, compete head to head, and rewrite their code between rounds. Football is continuous, multi-agent, and hard to reduce to one correct next move, which makes it useful for evaluating agents that have to act inside changing environments.

Models will keep advancing, but companies still need to know whether those models made their own systems better, worse, or merely different.

Everything’s an Eval

Evals are how companies understand, control, and improve AI systems once those systems start doing real work.

Benchmarks still matter, but the center of gravity is moving toward private verification: tests built around a company’s own workflows, standards, data, and judgment.

As Satya Nadella puts it:

“In consuming intelligence, you are creating intelligence. And what you create should belong to you. This is your particular intelligence, in Hayek’s sense: the knowledge of time, place, and circumstance that no one else can hold. It knows what you think, what you value, and how you measure success…”

Create your private evals, because evals define what “good” looks like inside the organization. Also, retain ownership of your organization’s memory, traces, feedbacks, decisions, and institutional context, and ability to use outputs of models from your own tasks and queries.”

As agents become more capable, the firms that benefit most will be the ones that can tell whether their systems are actually improving.

Evaluation therefore has to become part of operating the system, making AI legible enough to trust and measurable enough to improve.

In other words, everything’s an eval.

“We’ve been roughly right on technological predictions and pretty wrong on the social and economic implications…. …our industry underestimated how much we’re going to be able to keep people at the center of everything.”

Less than a week later, the latest in an emergent pattern of SF-based, AI-pilled intellectuals writing long, narrative (often negative) predictions on what’s to come for AI was published: AI2040.

Presented as the successor to AI2027, it’s a detailed future scenario and policy recommendation in the form of an interactive paper/website from the AI Futures Project.

Almost all of these predictive narratives, however, ignore what I refer to as the “messy middle,” where reality actually tends to unfold. The scenarios typically assume intelligence alone stands between now and the future, so more of it gets us there faster. But in the messy middle, intelligence meets the friction of the real world: institutions, supply chains, physical constraints, and the public.

Disregarding the messy middle is largely why many of the social and economic predictions Altman mentions about AI have thus far been misguided: it’s difficult to predict complexity.

As George Hotz wrote:

“I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. Then I joined the real world and actually tried to do things…You cannot take over the world with tokens.”

The Messy Middle

Consider what’s already happened to many of the headline predictions from ~12 months ago. Confident near-term predictions of the past year took contingent, middle-heavy situations and sold them as clean conclusions.

OpenAI itself was created (as a non-profit, no less) because its founders feared that Demis Hassabis (AI good guy Demis Hassabis), would become an “AGI dictator.” Google developed the Transformer, but OpenAI figured out what to do with it.

For several years thereafter, OpenAI was the clear winner, and then Anthropic (founded by an OpenAI defector) released Claude Code and ate its lunch. Now Anthropic is back on its heels after clashing with the government and a disastrous public rollout of Fable 5, and OpenAI is back in action (now as a for-profit entity) with GPT-5.6 Sol.

Alex Karp went on CNBC and excoriated the frontier labs over “data sovereignty,” “zero data retention” and their “stealing” critical IP from their users based on their prompts and context. Amidst this backlash, both companies are scrambling as they prepare for IPOs while their customers begin evaluating the ROI of their token spend and delegate more tasks to cheaper, good-enough open weights models.

The Wall Street Journal reports OpenAI is even considering offering the US Government a ~5% stake in its for-profit division, ostensibly for competitive, economic, or political reasons.

“One reason may be to buy political advantage against competitors. OpenAI faces competition from the likes of Anthropic, Google, Meta and xAI. Unlike its top rivals, OpenAI doesn’t have a large pool of cash or profits on its balance sheet to finance its data center build-out.

OpenAI plans to spend $600 billion on AI infrastructure by 2030, yet it is generating about $2 billion in revenue a month. An IPO would raise cash, but nowhere near enough to finance its ambitions. Perhaps Mr. Altman hopes that giving the government a stake will lower his company’s cost of borrowing.

Government ownership might also yield regulatory favors—such as faster approvals for models, permits for data centers or federal contracts. A government stake could provide an implicit backstop, making OpenAI too big to fail. The government won’t want to take a loss on its stake, even if it means keeping it alive like a zombie firm.”

To make matters even more complex, China itself had talks with leading AI companies about limiting foreign access to advanced AI models, both released and unreleased.

As it relates to compute, the entire world was supposed to be severely compute-constrained, and now SpaceXAI (lol) is selling compute to Anthropic, META is on record as considering a “cloud compute” business to sell excess supply, and AI giants are handing out tons of free computing power to grab startup share.

As noted earlier, despite having warned for years that AI could lead to a severe knowledge-worker catastrophe, rendering millions of white-collar workers obsolete, executives like Dario Amodei, Sam Altman, Mark Zuckerberg, and Andy Jassy are now walking back those claims. Writes the Wall Street Journal:

“Collectively, the narrative has shifted from worker-light doomsday scenarios caused by AI to a future in which workers keep their jobs—and get a productivity boost.”

Even mortgage-your-home-but-don’t-sell-Bitcoin Strategy CEO Michael Saylor is breaking his promise and “increasing his USD reserves” (read: selling $BTC).

In short, the world simply isn’t as straightforward as many of the armchair philosophers pontificating about the future of AI would like it to be, no matter how deeply they think about their predictive scenarios.

And none of this even begins to unpack the complexity of supply chains, manufacturing realities, raw material availability, geopolitics, and war.

What’s more, if society does decide that things are moving too quickly, and that AI is the scapegoat, there will be resistance. We’re seeing early signs of this manifesting already with Sam Altman’s home having been attacked, for example, and the first protests in front of the frontier labs’ offices being held this past weekend.

Unknown Unknowns

For a number of reasons, AI is a particularly difficult technology to understand deeply. It involves advanced mathematics, quantities of data that defy comprehension, and the industry’s leaders seem particularly ill-equipped to communicate anything to the general public.

It’s in part for this reason that these long narrative predictions are so popular: they’re sensationalist and engaging, but they’re very difficult to dissect and evaluate, let alone disprove. It should come as no surprise that the earliest examples of this genre were the most compelling, for we knew even less then than we do now about what AI would eventually be capable of and how it might impact humanity.

But the problem with these predictive exercises is the inherent entropy in the system and what Donald Rumsfeld called the “unknown unknowns.” As Rumsfeld wrote himself once he left office:

“The idea of known and unknown unknowns recognizes that the information those in positions of responsibility in government, as well as in other human endeavors, have at their disposal is almost always incomplete.”

There’s a selection bias buried in all this uncertainty, too. The scenario writers reach for the unknown to justify alarm (and action), but if we genuinely can’t see what is coming, it’s as likely to be a breakthrough as a catastrophe.

Redirection

Nobody can forecast with any confidence what superintelligence looks like or the consequences it might have on humanity. We can, however, forecast what the world will look like if energy becomes cheap and human lifespans stretch by a decade or two. In fact, intelligence’s real dividend lies in compounding engineering and design capabilities aimed at physical problems. But perhaps that’s too evident, too boring, or at odds with the authors’ intent (which we’ll get to below).

Energy, longevity, and robotics, for example, are all set to explode in the next decade, especially as we learn to harness the compounding effects of AI. The intelligence explosion, on the other hand, is hard to forecast because its defining feature, runaway recursive self-improvement, has no precedent.

Commonwealth Fusion Systems is finishing a compact nuclear fusion reactor that should reach net energy in 2027, with a first commercial plant under contract in Virginia and a billion-dollar power deal already signed. The FDA has cleared the first human trial of cellular reprogramming, a therapy that resets the age of cells; in mice, the same approach has extended remaining life by more than a hundred percent.

In robotics, intelligence finally has to touch matter, and the part being built now is the software that lets a machine move at all. Dimensional ships an open-source operating system that sits between an AI model and a robot, so an agent can drive a machine in plain language instead of the years of bespoke integration the field runs on today. The bet is that as robot models commoditize, the durable position forms in the software plugging them into the physical world, the same logic I have argued about the models themselves. (Disclosure: Dimensional is a Nazaré Ventures portfolio company.)

In short, apocalypse and superabundance are sideshows. Hypothetical narratives draw attention away from the real future (the tangible, near-term stuff actually being built), and direct it towards an imagined future (superintelligence, takeoff, the switch) despite the former being both more knowable and more important. The main event involves energy, longevity, and robotics rather than some self-directed robot demi-god with its own desires.

The Prophet Motive

All of this is sitting in plain sight, so why is the oxygen going to extinction instead of abundant energy, longevity, and robotics?

AI2027 was mostly legible as fiction. People referenced it to demonstrate “[AI alignment] deserves attention,” but almost never literally as “this exact sequence is happening.”

AI2040 differs from most of the “narrative prediction” genre in its prescription. Despite numerous “disclaimers,” they treat the underlying assumptions as settled, which is precisely what I articulated as a mistake in my last essay.

San Francisco and the AI community echo chamber now appear to be in an ever-more-grandiose competition to diagnose problems, prescribe solutions, and predict what will happen to the world when the low-probability-but-high-impact scenarios they obsess over finally come true.

The trouble is that they know their genre is particularly compelling at the moment. They’re manufacturing backlash and calling it a warning. Why, then, do such smart people use hyperbole like species-level extinction when they claim they want to be taken seriously?

1. They believe it literally. The “Don’t Look Up” scenario. Nobody who thought extinction was imminent writes a branching interactive website with five selectable endings and footnotes about their timeline updates.

2. They believe some real tail risk and knowingly inflate the vividness and the certainty. Classic “millenarian framing,” because calibrated uncertainty doesn’t compel policymakers to act, but “we all might die” certainly does (especially if the public believes it). This is motivated epistemics: identify the imagery that works, then backfill the conviction.

3. They don’t really believe the scenario and deploy it as an instrument. For relevance, influence, funding, power, organizational survival, regulatory capture, etc.

You don’t need to adjudicate which is actually true, because it’s at least one of them, and it might be a bit of all three. Whichever it is, extinction is a premise that renders the prescription unquestionable because it transcends “practical” considerations such as cost, enforcement, and other “messy middle” variables.

Either way, it’s difficult to disassociate the substance from the objective, which is clearly some form of direct or indirect influence over what happens in the future. There’s a particular arrogance in deciding, on behalf of everyone, what the future holds and who is fit to be trusted with it. At least representative government officials are elected. The scenario writers (along with labs and their leaders, like Anthropic and Dario Amodei) have appointed themselves as the ones who see clearly, while the rest of us blunder toward catastrophe. By consequence, they see fit to propose that the world should pursue their prescriptions because they’ve “seen the future first” and know how this ends.

Prediction-as-Value

In October of last year, Alex Danco published a brilliant piece on Prediction: the Successor to Postmodernism.

It argues that “prediction” is the cultural movement replacing postmodernism: the master frame for how we make meaning, build businesses, and find purpose in the AI century. It’s thoughtful, well-written, and helpful for understanding the cultural foundations that beget things like financial nihilism, memecoins, and prediction markets and, in turn, how they relate to artificial intelligence. Of particular importance is the following passage:

How early or late you are to something is now an essential component of your relationship to that thing. The timelines and reels that represent “what is going on” are increasingly about a single meta-topic: are you predicting it, or is it predicting you?

Many of the narratives being published from the belly of the beast appear to desperately want to be the former, but upon further inspection are often the latter. Offering thoughtful, structured arguments about where the world is going and how AI fits within it is like a rite of passage that the most enlightened in the industry must endure time and again to solidify their legitimacy.

But it’s gone just a bit too far. The echo-chamber could use less extreme philosophizing and a bit more levity and humility. Hotz is helpful here again.

“AI 2040 includes this picture of a datacenter in the ocean. Just like vaporware, you can generate a picture easily. But in reality, you have to deal with supply chains. You have to deal with them shipping you the wrong part, the thing not meeting the spec, it randomly failing after 20 minutes, the chip warping in the reflow oven. Did you consider the barnacles?”

Steampunk represents a specific kind of futurism: the future imagined through the machinery of the present. I often think AI discourse is unwittingly channeling H.G. Wells, imagining the next century with today’s machines before the science of intelligence is fully understood.

You know the syllogism, because it’s currently organizing a few trillion dollars of capital: scaling leads to AGI, AGI leads to recursive technological acceleration, and the resulting superintelligence gets to work on the Dyson sphere, the star-scale energy harvester that has become the timeline’s favorite retirement plan for humanity. Indeed as AI 2027 predicts in the hopeful version of its 2 endings:

“The rockets start launching. People terraform and settle the solar system, and prepare to go beyond. AIs running at thousands of times subjective human speed reflect on the meaning of existence, exchanging findings with each other, and shaping the values it will bring to the stars. A new age dawns, one that is unimaginably amazing in almost every way but more familiar in some.”

and in the not so hopeful version where the robots wipe us out:

“The new decade dawns with Consensus-1’s robot servitors spreading throughout the solar system. By 2035, trillions of tons of planetary material have been launched into space and turned into rings of satellites orbiting the sun.”

The logic of scaling and its extrapolation might be sound, and perhaps this is the way. Either way, whenever we do achieve AGI/ASI and manage to apply it successfully, most predictions fall apart, as I said when I wrote about moats. But the last object in the chain deserves a closer look, because we have seen it before. It was a hansom cab, and it flew over Paris.

The Flying Cab

In 1883, Albert Robida imagined Paris in the 1950s with the aérocab: the horse cab of his own streets, lifted into the air. Le Vingtième Siècle also gave the home a téléphonoscope, a wall-mounted screen for news, theater, lessons, conversation, and family dinners at a distance. Sixteen years later, Jean-Marc Côté and a team of illustrators produced the En L’An 2000 cards for the 1900 Paris Exposition. They returned to the same assumptions: air cabs, winged firemen, automated domestic work, mechanized leisure, undersea transport. The cards were printed in batches from 1899 to 1910, never distributed, and forgotten until Isaac Asimov published a recovered set as Futuredays in 1986.

The cards clearly envision the year 2000 as an extrapolation of 1900. That instinct has a long pedigree in philosophy and cognitive science: imagination does not create ex nihilo, but recombines, transforms, and abstracts from prior experience.

Every era draws its future in the vocabulary of its present, and novelty usually comes from new arrangements of stored material: memories, learned concepts, metaphors, images, and technical forms already available to the mind. Robida almost perfectly predicted television, screens, and video, for example, but he built it out of brass and gaslight.

His téléphonoscope anticipates the future, but its imagined form still belongs to the materials and assumptions of his own century.

The Sixth Decimal Place

Why did the sharpest futurists of the belle époque imagine a future so mechanically inventive, yet so conceptually narrow? Because they lived inside a scientific culture that increasingly treated the foundations as settled.

In 1894, at the dedication of the Ryerson Physical Laboratory in Chicago, Albert Michelson told his audience that most of the grand underlying principles of physical science had been firmly established. He then quoted an unnamed eminent physicist: “The future truths of physical science are to be looked for in the sixth place of decimals.” The line is often attributed to Lord Kelvin, but Kelvin never said it. It kept circulating because it captured what many physicists already believed.

Incredibly, the same confidence shaped how senior physicists guided their students. In the 1870s, Philipp von Jolly advised the young Max Planck against a career in physics, describing it as a “nearly matured” science with only minor problems left to examine. In April 1900, Kelvin gave his Royal Institution lecture on the two remaining “clouds” over physics: the ether problem and the equipartition of energy. To many physicists, the basic structure of the field was already in place, leaving only marginal improvements for future scientists: refining existing theories, applying their principles, and engineering new uses.

That the belle époque could imagine such extraordinary inventions without considering new substrates speaks to how treating fundamentals as settled limits creativity and inhibits our ability to properly predict what’s to come.

The Break

The irony is that the foundations were actually evolving at the same time as the leading physicists were declaring them settled.

Between 1895 and 1905, the discovery of X-rays and radioactivity (both by accident), the electron, Planck’s quantum, and Einstein’s annus mirabilis papers had turned the two “minor” problems into the basis of a new physics. Kelvin counted his clouds in April 1900, but within five years, the ether problem had been dissolved and equipartition was on its way to being replaced.

This “new” physics, however, first arrived as a collection of awkward observations and unresolved technical problems. They weren’t treated as groundbreaking at all, because from within the old framework these discoveries were treated as “refinements” or marginal improvements, which were the only things left for science to explain.

The Côté cards kept appearing through this transition, for example. My favorite card, the one Asimov singled out, heats a living room with a speck of radium glowing in the fireplace. Although the physics that would define the twentieth century had already appeared, the old imagination could still only understand it as an improved source of heat. Electronics, nuclear power, computation, and lasers came out of the same anomalies that 1900 had treated as refinements.

Engineering Intelligence

Which brings us back to the present and AI. Few serious people claim to possess a complete theory of intelligence, but much of the industry has become highly confident in a method for producing its final form.

Transformers, scaling laws, reinforcement learning, inference-time compute; whatever remains is engineering. Sam Altman opened 2025 writing: “We are now confident we know how to build AGI as we have traditionally understood it.“ Two paragraphs later: “With superintelligence, we can do anything else.” The first sentence treats AGI as self-evidently attainable, and little more than persistent engineering. The second turns superintelligence into the general instrument through which all other problems get solved downstream. Together they constitute incredible confidence that the “foundations are settled.”

If he’s to be believed, Sutton’s Bitter Lesson has been promoted from essay to ideology, and AI researchers now get advised off working on fundamental architecture roughly the way von Jolly advised Planck off physics. We’re seeing early versions of this manifest in a declining number of students electing to pursue computer science degrees.

If you want to know whether the industry believes this, follow the money. The four largest hyperscalers have told investors to expect roughly $700 billion in capital expenditure for 2026, up from about $410 billion in 2025.

And to be fair, the confidence has earned its footing. A sixty-year-old in 1900 had watched railways, the telegraph, the telephone, and electric light arrive within a single lifetime, and extrapolated accordingly. A researcher today has watched eight years of scaling curves hold and every predicted wall get climbed. The labs have built something extraordinary and the exponentials are real (the valuations are a separate conversation). The risk is that a record of recent progress starts being treated as a law of the future, disqualifying any other pursuits and limiting our imagination.

The Anomaly Bin

Every confident era has a way of classifying the problems that do not yet fit. Ilya Sutskever presented on the main stage at NeurIPS in December 2024 and confidently claimed pre-training as we know it would unquestionably end: “While compute is growing,” he said, “the data is not growing, because we have but one internet.”

Although Ilya was being a bit hyperbolic, the unresolved problems are familiar. Current models do still struggle with the basic properties we associate with intelligence: continuous learning, sample efficiency, energy efficiency, and reliable long-horizon action.

Models freeze at deployment and cannot learn from experience. Sample efficiency runs orders of magnitude short of a child. Brains produce intelligence with tiny amounts of power compared with AI systems, yet the industry is currently solving the problem by building enormous energy infrastructure. Our below-the-model thesis, for example, starts with the fact that the current path to machine intelligence is becoming a physical infrastructure problem.

At the same conference, however, Noam Brown said he had never heard a serious AI researcher say the field is hitting a wall. Sutskever and Brown can both be right: AI can keep advancing while still accumulating unresolved anomalies. But the existence of those anomalies doesn’t disprove the current paradigm, and the success of the current paradigm doesn’t make them irrelevant. In 1900, the loose ends in physics still looked manageable from inside the old framework. The question for AI is whether today’s loose ends are manageable defects in the current approach, or early signs that the next foundation will look different.

The Dyson Sphere Is a Flying Cab

The Dyson sphere belongs to the same pattern. Freeman Dyson’s 1960 paper in Science framed it as a SETI thought experiment, borrowing the idea, by his own account, from Olaf Stapledon’s 1937 novel Star Maker. Nikolai Kardashev’s 1964 scale did something similar, ranking civilizations by the power they consume. Both ideas come from a high-industrial imagination in which progress still meant larger systems of energy capture, and the contemporary version carries that assumption into AI: if intelligence is produced by scaling computation, then the endpoint naturally becomes a civilization-scale energy project. The question is whether intelligence will remain bound to that path. If it becomes more efficient, more adaptive, or less dependent on today’s infrastructure, the Dyson sphere starts to look like this era’s téléphonoscope: right about the ambition, wrong about the substrate.

The analogy only works if it survives the obvious objection: Michelson’s generation was wrong almost immediately, while the scaling generation has been directionally right for years. Most anomalies resolve inside the framework that produced them; Kelvin’s clouds are famous because they became exceptions. Einstein himself dissolved the ether problem by showing that light didn’t need a medium, then spent decades resisting the quantum theory that followed from the same rupture in physics. He knew the theory worked and still couldn’t accept what it implied. Correctly seeing one break in the foundations didn’t make him a reliable guide to the next.

We won’t know until much later whether the current path was sufficient, but the industry is already allocating roughly $700 billion a year as if it were. Our fund is built around the practical consequence of that uncertainty: the layers above and below the model keep their value whether the current recipe holds or breaks. I made the longer version of that argument in Models Aren’t Moats.

Robida’s téléphonoscope anticipated a real future but built it out of the machinery of his own century. Steampunk shows a future that looks radical until you notice it’s built on the assumption of the present extrapolated forward. The future will inevitably be transformative, but it will probably arrive in forms we can’t anticipate.

They are remodeling themselves to look like what is driving the change in the first place.

AI is itself a learning machine, it spawned its own adaptive user (agents), and the smartest way to remake companies and institutions is to rebuild them as AI-powered learning machines too.

This unifies two arguments from earlier in the Robot Wave arc:

1. Models Aren’t Moats: enduring AI-native businesses are the ones building specialized intelligence around the model, compounding their value as the frontier improves beneath them.

2. The AI-Enhanced Operator: humans who understand AI are the smallest indivisible piece of specialized intelligence. These humans are thus in high demand but in short supply because mastering AI is impossible (the target never stops moving) and requires a unique temperament for working permanently at the edge.

Building enduring AI-native businesses means finding AI-Enhanced Operators and giving them as much leverage as possible to drive business impact. It also means structuring the organization itself as a domain-specific AI-powered learning machine.

The Machine Before the Machine

You might say companies have always, in some sense, been learning machines. Capitalism and market dynamics are self-correcting mechanisms designed to respond to a reward function.

Take it from Markie Wagner, who herself is building a self-proclaimed learning machine:

“Capitalism is organizational evolution. Millions of businesses compete in the marketplace with offerings that they think customers will want. Some thrive and grow. Others die. Each company evolves, too. People come and go. An experiment becomes a process, a process becomes a web of tacit knowledge. Products are introduced, and products are retired.”

Citadel, for example, has been one of the most ruthlessly successful learning machines for the better part of thirty years, growing into one of the biggest hedge funds in the world, now managing $68 billion (more than $19 billion of it Griffin’s and his colleagues’ own money) and delivering a 19% average annual net gain since 1990 with only two losing years. By net dollar gains since inception, LCH Investments ranks it the most profitable hedge fund in history.

The fund’s durable advantage is the apparatus it has built to improve as it learns over time. The New Yorker describes Griffin’s mission as updating the firm’s strategies and infrastructure so relentlessly that its edge holds for decades.

“His mission has always been different: to build finance businesses that update their strategies and infrastructure so relentlessly that they beat rivals not just today but over decades. Paradoxically, maintaining a consistent edge requires constant, unsentimental internal change—of processes, technology, and people. Citadel takes ideas that are just beginning to circulate and improves them, with math or technology or data that others haven’t thought to use.”

This famously extends to the firm’s culture and org chart, too. Although not the first to pioneer the “pod” structure, Citadel helped make pods one of the defining organizational forms of modern equity investing: small, sector-specialist teams operating within a centralized system for risk and capital allocation. Each team operates autonomously and is judged on performance. Top performers get more money and underperformers get cut.

Now, a learning machine is only as good as its reward function, and finance happens to have a great one: profit and loss. Implicit within that reward function is a common unit by which everything is measured: the dollar. The dollar collapses heterogeneous, incommensurable outcomes onto a single ordered axis. It’s fungible, divisible, transferable, and exogenous. The dollar is why P&L works as a reward function. Citadel’s efficiency as a learning machine thus comes from three things:

1. A verifiable reward function: P&L within a given window. The environment hands back unambiguous, fast feedback, in a unit that lets you compare every desk, strategy, and trader on one axis. The signal is clean and strong.

2. Ken Griffin & elite talent: An elite operator orchestrating the machine with the judgment to act well on the signal, and elite individual managers improving the whole.

3. The organization: a collective designed by the operator, built to run iterative loops against the reward function: allocate more capital to what works and cut what doesn’t. The gains compound. A good reward function is wasted without a human to build a structure that properly learns from it.

Citadel’s useful as an example because it shows how successful a high-performing learning machine can be when it satisfies three core criteria: a good reward function, an operator(s) willing to act on the signal, and an organization built to metabolize feedback.

AI makes that structure more broadly available, and makes becoming a good learning machine more valuable. By making parts of the learning process cheaper and faster to run at scale, it lowers the cost of the basic learning loop: try something, observe what happens, compare it against the goal, and adjust.

Signals and Sensibility

But more experimentation doesn’t necessarily mean more learning. A system only learns when it knows what good looks like. That was hard before AI, and it will remain hard after AI.

AI is often strongest where the reward is verifiable: math, code, games, formal logic, etc. LayerLens has chronicled how long games have served as a proving ground for machine intelligence. (Disclosure: LayerLens is a Nazaré Ventures portfolio company.) Reasoning models improved in part because reinforcement learning from verifiable rewards (RLVR) satisfied this condition: output was evaluated against answers known to be correct.

The same dynamic also explains some of the “jagged frontier” that makes AI more difficult to understand. AI doesn’t improve evenly across everything humans consider difficult because it improves fastest where attempts produce feedback the system can actually use.

Anthropic and OpenAI know this, too, which is why they’ve both optimized for programming with Claude Code and Codex. Code follows a set of observable, verifiable rules, meaning AI can do it well and improve extremely rapidly. It should come as no surprise that coding harnesses have become the first major commercial market for agentic AI.

The broader problem is that AI applied to the world writ large (especially to work and jobs) doesn’t work as well as it does in software. Real life doesn’t have a “common unit to compare outcomes.” But that hasn’t stopped folks from trying to find one, though, as Markie Wagner articulated in the same essay as above:

“The promise of AI is that it will turn businesses into software so that they can evolve over millions of tiny iterations. Beautiful, ideal, complex things can only emerge as the result of tremendous trial and error over time.”

Applied AI works best when the job in question can be made legible to software. “Turning businesses into software” thus means identifying which parts of the work can be made legible, then structuring them so AI can accelerate the learning loop: attempt possibilities at scale, observe, compare outcomes, and improve.

But this, it turns out, is hard: most jobs aren’t naturally legible to software.

As we’ve written before, the frontier labs Iknow this, which explains why they feel required to invest hundreds of millions of dollars into “enterprise services companies” embedding “FDEs” into businesses to teach folks how to use their products.

Although it might be clear that AI is going to transform many, many things, it is not yet clear how. Take the tokenmaxxing frenzy, for example.

Tokenmaxxing was an early attempt to invent a common unit, and it failed because it confused usage with value. Tokens are easy to count, easy to compare, and widely available (did someone say commodity?), which made them tempting as a proxy for productivity.

But tokenmaxxing only tracked consumption, not what that consumption represents. It lacked any causal relationship to productivity, value, or material business impact, making it useless.

This is my whole point in Code Isn’t a Coup: the models are outstanding (and improving), but we’re a long way from life-or-death outcomes because of the models themselves. AI is, at the moment, a paradigm-shifting tool. That tool definitely warrants reorganization around it, but it’s not yet an outcome in and of itself.

What Good Looks Like

Which brings us back to humans. Contrary to the headlines and popular belief, we’ll need a lot of them, because although AI can do a growing share of the work downstream of a reward signal, it can’t do what matters most.

Tokenmaxxing failed because it measured usage instead of value. The important human role is defining the value side of the equation: what’s the business logic? How do we measure it? What are good representative proxies? How might we make parts of what we do legible to this incredible new tool?

Finance had its common unit handed to it. Most domains do not, so the operator has to build one: a local, defensible proxy that stands in for value where no universal measure exists. AI cannot build that proxy for you because the system optimizes only the unit it is given.

That kind of definition is useless if it arrives after the fact. All the hallmark buzzwords of the moment (taste, judgment, vision, etc.) have to inform the loop before AI starts generating outputs, because the system will improve whatever the process teaches it.

People who understand AI, their business, and their customers have to be closer to the design of the loop itself. This is the peak application of the AI-enhanced operator’s skillset.

Because AI creates new kinds of work before anyone knows how to evaluate them, the operator embraces ambiguity and discovers what good AI-powered work looks like by doing it, compounding their own learning in the process. They author and audit the reward function, then evaluate the result and repeat.

Unfortunately, too few people can do this well at the moment, which is why everyone’s reaching for the word “agency.” Credentials and experience matter less than temperament: curiosity, tolerance for discomfort, strong opinions loosely held, and the willingness to constantly update your worldview. Identifying these rare individuals is where AI-native value begins to compound.

New Form of the Firm

Rem Koning recently published a study on what makes a firm “AI-Native.” He distinguishes between what he calls process and product channels. In his words, the process involves using agents like Claude or ChatGPT to move faster with a lean team, whereas the product channel sells AI that does work a human used to do.

According to Koning, process changes don’t reorganize the firm nearly as much as product changes do, because the economics of building a company around an AI product require a fundamentally different structure. He concludes that AI startups are smaller, flatter (half a layer fewer), more engineer-heavy, more senior, have fewer managers, and are more efficient per employee (by valuation).

[Aside: Koning also asserts that smaller firms do not mean fewer jobs. There’s a surge of new-firm entry, meaning that the increasing number of smaller firms balances the fewer number of jobs per firm.]

Practically, this means that companies earnestly trying to prepare themselves to be “AI-native” now have to do three things at once: identify their “operators,” give them more leverage, and rebuild the organization so it can learn from both endogenous and exogenous change.

Inside the business, more work has to become legible to software. Outside the business, the company has to keep adjusting to the frontier of new models and emerging agent capabilities. Customer expectations and constraints keep shifting too. The organization becomes a learning machine when it can absorb both kinds of information in real time and change its behavior accordingly.

The old paradigm organized companies around narrow human roles, then added management layers to coordinate the work between them. AI puts pressure on that arrangement because capable individuals can now do much more, while software handles more coordination.

Jack Dorsey and Roelof Botha articulated as much in a recent memo on hierarchy as an information-routing system. As more information moves through software and more execution moves through AI-mediated systems, some of the routing work that justified managerial layers becomes unnecessary. If the company wants to learn from endogenous and exogenous change quickly enough to react successfully, it has to reduce organizational drag between the people who understand what should change and the systems capable of changing it.

Brian Armstrong was pilloried for rebuilding Coinbase “as an intelligence, with humans around the edge aligning it,” but he’s probably right to do so. He recasts the company as a system that decides and acts on its own, with humans at the boundary where their judgment matters most.

AI expands what capable individuals can do and shifts more execution to software. Both weaken the coordination work that justified the old org chart. People are actually more important in this paradigm because companies will rely more heavily on their operators, but each individual organization will require fewer of them.

Much of the discourse about AI’s impact on jobs misses the point. It’s easy to claim superabundance or apocalypse, but both are extremely reductive. Although there will probably be an uneven and perhaps turbulent transition, AI-native companies need people who can steer the business as the frontier moves. But again, those people are rare.

Just like the industry is moving more quickly than the institutions tasked with regulating it, the systems in place to train the AI-native workers are moving more slowly than the companies trying to hire them.

Mirror Image

Companies, however, are just the beginning: responding first to change because the market forces them to. As we mentioned earlier, capitalism is one of the most efficient meta-learning machines we have.

But society metabolizes change more slowly because humans are at a speed disadvantage: our habits and institutions have long half-lives. The jobs debate is but an expression of what I’ve called the “Too Fast Threshold.” AI will eliminate some jobs and change many more, but the deeper shift is the spread of a kind of as-yet-undefined work wherever AI diffuses. If AI is inevitable, and I firmly believe it is, companies reorganizing as AI-powered learning machines is only the first step.

Like a lot of debates about AI, the direction is clear even if the resulting consequences are not. We will still need humans, but we will need them to do different things. Individuals can adapt for themselves but societies have to build institutions that help more people adapt at scale.

One of our next great challenges as a society is learning to produce enough people who can work within the AI reorganization. If AI changes the way we work, hustle’s not enough. We’ll need systems that help people understand this new technology and learn to work with it.

John F. Kennedy recognized a similar dynamic in 1961, signing a solution into law in ‘62. At the time, automation was changing the labor market faster than many workers could adapt. Rather than just studying the problem, regulating automation, or flat out waiting, his administration decided to train people for the work the economy would need. He is famously quoted as saying:

*“The unemployed whose skills have been rendered obsolete by automation and other technological changes must be equipped with new skills enabling them to become productive members of our society once again.

Large scale unemployment during a recession is bad enough, but large scale unemployment during a period of prosperity would be intolerable.”*

It is safe to say that AI will usher in an unprecedented period of prosperity, at least for those well-positioned to benefit from it. Despite capitalism’s ruthless efficiency, the answer to AI can’t be left entirely to the market if we want this to go well.

But institutions face the same problems the rest of us do. They typically develop systems to train people for existing work, but AI keeps creating and redefining work in real time. For better or for worse, the response probably has to resemble a learning loop too.

Regulation has the same defect. A rule is a fixed answer to a question the world has often already moved past. Legislators write for the conditions in front of them, and by the time a rule takes effect, the conditions have shifted. This is why Kennedy’s instinct holds up. He could have tried to regulate automation directly. Instead, he built a process to keep retraining workers as the labor market turned over, which proved far more resilient than a static rule would have. The regulations that survive a fast frontier work the same way. They build in their own revision, with a schedule for review and a live channel back from the people they govern, so the system can tell whether a rule still does what it was written to do.

I’ve written before that AI conjures the religious imagery of old: the Tower of Babel, Prometheus stealing fire, the Creation of Adam, and God creating humans in his likeness.

But AI may be so uniquely recursive as to complete the circle: we may be building “intelligence” in our likeness, only to find ourselves reorganizing our companies and institutions around its image. Our working lives change to match.

The frontier AI industry is both extremely concentrated and dominated by private enterprises. The most powerful artificial intelligence in the world is built by only a few companies, in a few buildings, in a few countries.

On June 12, the US government ordered Anthropic to suspend access to Fable 5 and Mythos 5, its two newest frontier models, for any foreign national. Because citizenship cannot be verified across millions of users in real time, the only compliant move was to switch the models off for everyone, and they were gone worldwide within hours. Opus 4.8 and the older models kept running.

As of writing the dispute is unresolved and the two models are still dark, the clearest demonstration yet of how fragile the average user’s access to frontier AI has become.

Before this, using Claude or ChatGPT was as simple as paying for a subscription or an API key. There were whispers of “Sovereign AI,” or using “open weights” models running on local machines, but most people just used Claude, until their access was revoked.

By now it’s clear what’s at stake: access to the most powerful artificial intelligence in the world. Because of the industry’s concentration and composition, access can be modified or revoked at any time, even to paying customers.

As we’ve written before, control typically only matters once it’s lost, and in the wake of this controversy, individuals, companies, and countries are looking to design robust AI stacks that don’t depend on continued access to a frontier lab.

Frontier AI is and will remain concentrated because training requires enormous, visible, governable infrastructure. But useful AI is becoming less dependent on hosted access, because inference, weights, and training are each beginning to escape the frontier labs in different ways.

Why Control Concentrates…

Frontier concentration starts as an engineering requirement. The best models are thus far often the largest models, and training them requires tens of thousands of accelerators in the same place, wired together with enough bandwidth to minimize latency and synchronize the whole cluster on every step.

This kind of enormous clustered hardware is also extremely expensive, which is why the frontier labs are some of the most capital-intensive businesses in history. Together they form a loop: you need capital to build the models, but you need good models to raise the capital. Only a handful of companies satisfy both conditions, which is why access to the frontier is permissioned.

The leading labs keep the parameters closed because access to the model is the product. Releasing the weights means there’s nothing left to sell, so the labs don’t make the parameters of their models available or downloadable. Users can’t own their own versions of Claude or ChatGPT. Instead, these models are made available as a service users can pay for using accounts, subscriptions, or APIs.

Access sold as a service can be throttled, revoked, or switched off, all of which happened to Fable.

…and Where the Cracks Are Showing

The same requirements that concentrate the frontier are now making it harder to expand, for reasons that have nothing to do with algorithms.

Training clusters are scaling toward the gigawatt range. xAI’s Colossus is moving from 350 MW toward 1.5 GW and OpenAI’s Abilene site is expanding to 1.2 GW, with projections of 10 GW runs by the end of the decade. The grid as currently designed cannot keep pace.

But the bottleneck isn’t just available power generation, although the specialized equipment required to expand the grid is back-ordered.

Perhaps more importantly, the towns are voting the buildings down, even though only about 8 percent of Americans live near one. In the first quarter of 2026, community opposition blocked or delayed data center projects worth about $130 billion, the largest single quarter on record, with active opposition groups more than doubling to 833 across 49 states. New York’s legislature passed a one-year moratorium on large data centers, awaiting the governor’s signature, and Maine passed one before its governor vetoed it (and Europe is way ahead of the US on protests).

Clearly none of this halts the buildout or reduces demand for frontier models, but it raises the cost and slows the pace of development. Many people will continue to demand the best available models, but viable alternatives are improving and getting cheaper.

These alternatives center on three basic questions:

1. Can you own the model?

2. Can you run the model yourself?

3. Can new models be trained without the incumbents?

Open Weights

Start with ownership. Can a user possess the model at all? Access control with Fable is a case in point: when access is the product, access can be withheld. With open weights, the trained parameters are released for others to download, copy, and serve independently.

Possession would matter less if it required accepting a large drop in capability. For years, open models consistently underperformed the frontier such that they weren’t capable enough to rely on for anything meaningful.

Increasingly, that’s no longer the case. The best open models are now close enough in capability to the frontier that they’ve become practically useful, and for much cheaper than the frontier.

The numbers now support that shift. The Stanford AI Index put the closed-model lead near 3.3 percent in early 2026, and on individual tasks the best open models have already drawn level. Z.ai’s GLM-5.2, a roughly 750-billion-parameter mixture-of-experts model with a one-million-token context and an MIT license, scores 62.1 on SWE-bench Pro against GPT-5.5’s 58.6, at about a sixth of the cost, while still trailing Opus 4.8 on most coding head-to-heads. DeepSeek, Qwen, Kimi, and Llama sit in the same tier.

The practical point is simple: most work doesn’t require the frontier in the first place. It only requires a model good enough to clear the task’s threshold. Once an open model is good enough for the task, the marginal improvement in capability of a closed frontier model isn’t worth as much as control over the model itself. Open weights models have now definitively hit that threshold, and they continue to improve as well on more challenging benchmarks.

The same distinction applies to jurisdiction. Running GLM through Z.ai’s cloud subjects the traffic to China’s National Intelligence Law, which may be as unattractive as routing through the US labs. But downloading the same weights and serving them yourself means you’re beholden to no one. The model may be identical, but the dependency is different. In addition, services like Venice.ai offer hosted versions of the open-weights models, with some promises of privacy.

Open weights therefore provide control in ways that closed models can’t, but they don’t remove every dependency.

A downloaded open weights model still has to run somewhere, and if it runs through another hosted endpoint (as in the GLM Z.ai cloud example), the user’s only shifted the dependency to a different provider. The question of independent training comes later. First comes the more immediate problem: once the weights exist, can the user run them without a provider?

Running Models Locally

If open weights mean a user can own the model, local inference means they can use the model without a provider.

For most of the past few years, using a capable model meant sending a request to a provider’s data center, where the provider supplied the compute, saw the prompts, and controlled continued access.

That is changing because local hardware has become strong enough to run models that recently would have required much larger systems. NVIDIA’s DGX Spark, roughly the size of a large paperback, pairs a GB10 Grace Blackwell chip with 128 GB of unified memory and NVFP4 four-bit precision to run models up to around 200 billion parameters locally, or roughly 405 billion with two units linked over a 200 Gbps interconnect. Apple’s unified-memory machines do something similar through the MLX stack. Aggressive quantization makes much of this possible by reducing the memory required to run large models, albeit at the cost of some loss in quality.

When the weights run on a user-controlled machine, the model can run “locally” without an internet connection, so use no longer depends on a live provider. There is no provider logging prompts, no hosted endpoint that has to remain available, and no subscription required for the model to keep working. Continued access, therefore, is no longer mediated by an intermediary.

There is, however, a limit. Users can own open-weight models after they have been trained and released, and they can increasingly run those models locally on machines they control. But as discussed earlier, developing new frontier models, open or closed, still depends on the concentrated training infrastructure that narrowed the industry in the first place.

Whereas open weights and local inference reduce dependence once the model exists, training determines who can create the next one, which is why it remains the hardest dependency to break.

Training Without the Giants

The last dependency is training itself: can new models be built without the companies that own the largest clusters?

Training is the part of the stack most tightly bound to concentration, but it is not monolithic, and the costs split sharply depending on which half you mean.

Pretraining a base model from scratch is still the concentrated, frontier-capex half. Post-training, which now drives much of a model’s capability, has detached from that scale. Prime Intellect (a Nazaré portfolio company) has shown reinforcement learning at trillion-parameter scale on open mixture-of-experts models: with its open-source prime-rl stack it post-trains GLM-5 on long-horizon software-engineering tasks at 131k context, with sub-5-minute step times, on 28 H200 nodes, and the same recipe runs on Kimi and Nemotron.

Roughly 224 GPUs is a cluster a neolab or a well-funded company can rent, not a hundred-thousand-chip buildout. The step the labs assumed was their moat, the reinforcement learning that turns a base model into a capable agent, now runs on modest compute with open code. Reinforcement learning also distributes more naturally than pretraining, because the rollouts that dominate its cost run asynchronously and in parallel rather than locked in lockstep across one fabric.

The dependency does not disappear; it moves up a layer. Post-training runs on top of an existing open base model, and those still come from a handful of labs with frontier pretraining clusters, most of them Chinese. Cheap open post-training is real, and it rides on someone else continuing to release a frontier-scale base to build on.

Building that base without the giants is the harder, slower frontier. Covenant-72B, a 72-billion-parameter model from the Templar team, was trained across about seventy permissionless peers over commodity internet, with no whitelist and no central operator, using SparseLoCo to compress communication by about 146 times and a trustless validation scheme to keep anonymous contributors honest, at 94.5 percent compute utilization. It scored 67.1 on MMLU, slightly ahead of Llama-2-70B. That shows decentralized pretraining works, but lands years behind the live frontier.

Epoch AI judges it likely to be feasible at frontier scale in the future and suggests it is growing at nearly 20 times a year, four to five times faster than centralized training is scaling, while still unlikely to gather frontier-level compute this decade. The sign that this is becoming infrastructure rather than a stunt: Prime Intellect now sits in the NVIDIA Nemotron Coalition, contributing post-training infrastructure and 2,500-plus reinforcement-learning environments, with Nous Research and Pluralis training by the same logic.

Will the Open Stack Reach the Frontier?

The open stack can probably reach last year’s frontier, and on post-training it is already there. Reaching the live frontier is harder, and the obstacles are pre-training scale, private data, and funding rather than post-training compute.

“The frontier” can mean two different things. If it means the best model from 18-24 months ago, decentralized training is close enough to take seriously. Money is now the binding constraint: can these projects keep paying for large runs once the coordination problem looks solved?

If it means the best available model at any given point in time, the answer is probably not this decade, and possibly not ever. Distributed training can only gather so much spare compute, and the leading labs are still accumulating more.

There’s also recursion to take into account: the frontier improves every year in part because the labs use their own models to help build the next ones, so the pace of improvement quickens.

The strongest case for distributed training is efficiency. The compute required for a given level of capability keeps falling, and DeepSeek trained a competitive model for roughly a twentieth of a comparable budget. If that continues, decentralized training won’t need to match the frontier’s raw compute directly, because the amount of compute needed to reach a useful level keeps falling.

The harder limit is everything that has not come loose. The base models still come from a few frontier pre-training clusters, and the frontier’s edge increasingly sits in proprietary interaction data and in reward design for fuzzy, hard-to-verify work, where you cannot simply check the answer. Open post-training bites hardest in verifiable domains like code and math, and least where the reward itself is the moat.

Funding is also unresolved. No legitimately durable way to pay for community-scale training has emerged, so even if the technical path works, the money for the largest runs may never materialize.

The Common Property

Open weights, local inference, and distributed training are at different levels of maturity, but they share the same basic property: they reduce dependence on permission from a frontier provider.

The frontier will always be important, but it may also become inaccessible, illegal to use, or nationalized outright. As long as they survive, the largest labs will preserve advantages in capital, infrastructure, data, talent, and post-training, and many users will keep paying for the best hosted models when the work justifies it.

But the hosted frontier no longer has to be the whole stack. A company, country, or individual can use frontier access where it’s most valuable while keeping enough capability outside the hosted channel to survive disruptions in service.

After Fable, that’s the standard a serious, sovereign AI stack has to meet: when a provider stops granting access, useful AI must live on.

I can’t say I’m surprised. On April 29th I described three tiers of intelligence forming: a frontier the labs keep for themselves, productized commercial models in the middle, and open-weights models at the bottom.

Governments, I wrote, would demand the frontier “primarily for military and national security reasons, which they’ll eventually be granted whether the labs do so willingly or not.”

More Than Moral

I have championed open technology for years. I have invested in it, built it, and written about it.

Open technology often gets invoked as morally superior to closed technology, but morality is not why it wins.

I’ve seen this firsthand: principles don’t drive adoption. Users won’t choose open technology if the product isn’t better, and slogans like “open technology is better for humanity” aren’t business strategies. Open technology cannot endure exclusively for moral reasons. It must also add value.

Thankfully, it adds value because it’s cheaper, more accessible, more controllable (”sovereign”), and more resilient under constraints. When constraints aren’t the priority, these qualities are usually ignored. But the moment constraints become apparent, “good enough” open alternatives come roaring back into focus.

I called this dynamic MACHA, Make AI Cheap Again: if the first wave of AI was about capability, the next is about efficiency. In a MACHA world, “good enough” compute is a feature, not a flaw. I made the case that increasing economic pressure, geopolitical tensions, and a never-ending battle for control (all different forms of constraint) would end AI’s “scale at all costs” era and force a turn to cost-per-token discipline.

Sam Altman has conceded that token costs went from a non-issue to "a huge issue" inside a single quarter, and enterprises now route the bulk of agent work to cheap open models, reserving frontier calls for the hard problems and cutting costs by as much as 95%. If the gap between closed and open is that wide, someone will inevitably turn it into a business model.

Free AI is ending and tokenomics is beginning: we're now squarely in a MACHA world.

Who Holds the Switch

The Mythos suspension exposed an asymmetry: a closed model can be revoked, but an open one can’t.

Washington could switch off Mythos and Fable because they were gated, running through a single provider. No one can switch off DeepSeek, Qwen, or what’s left of Llama. Once the weights are out, they sit on ten thousand drives that no government directive can reach.

Only open models whose weights are transparent and verifiable offer control over the corresponding intelligence. Anything short of that involves trusting the provider, and as Anthropic demonstrated: trust takes a lifetime to build and an instant to break.

Dean Ball warned that training a soon-to-be-smarter-than-you system to deceive its own user is different than a model that refuses out in the open. Christian Catalini looked further down the road, to the decentralized answer a single chokepoint tends to summon:

From Anthropic’s original choices to Washington’s reply, the entire episode was an argument about who decides.

Back in April, when Mythos was announced as too dangerous to release, AISLE found that small, cheap, open-weights models recovered most of its showcase vulnerabilities, including the flagship FreeBSD exploit, caught by a model with 3.6 billion active parameters at eleven cents a million tokens.

The moat, they concluded, is the system, not the model. The week Fable shipped, OpenRouter combined a panel of budget models to achieve performance within one percent of the frontier on a research benchmark at half the cost. You can, indeed, revoke access to a model, but its capability is already available, and a weekend of open tooling can put most of it back.

Superabundance or Apocalypse

It’s worth noting that the US Government’s action is an immediate consequence of the hyperbolic debate surrounding frontier artificial intelligence: according to its leaders, AI either leads to superabundance or apocalypse.

Neither scenario is actually plausible, but it’s commonly employed rhetoric because it’s easy to communicate, easy to understand, and it animates both the public (which is useful to politicians) and investors (which is critical for fundraising).

Anthropic has portrayed Mythos as a dangerous, weaponizable model since its inception, and in a matter of days the US government agreed to treat it as such.

I’ll grant that the danger is real. Simon Willison spent two days with Fable and called it relentlessly proactive: give it a goal and it invents tricks nobody has written down to reach it, treating every constraint you set as a problem to route around. That makes it enormously useful to someone with a hard problem to solve, and dangerous in the hands of someone who wants to cause harm.

If you believe a model can meaningfully uplift a bad actor, gated and revocable access starts to look prudent. But the prudence and the power grab are the same. The question was never whether the technology is dangerous. It was who gets to hold the off switch, and whether they will keep their word. Concentrating the switch doesn’t make the danger go away. It moves it from many hands to very few, and leaves one company, or one government, deciding what billions of people are allowed to compute. That is the larger hazard, and it is the one open weights answer.

The Unwilling Hero

A word on Dario Amodei. There is a scene in Gladiator where a dying Marcus Aurelius offers Maximus the chance to be emperor of Rome.

“There is one more duty I must ask of you before you go home,” he groans. “I want you to become the protector of Rome after I die…Do you accept this great honor I have offered you?”

“With all my heart, no,” replies Maximus.

“That is why it must be you,” Aurelius asserts.

This is the image of an unwilling hero, popularized as the refusal of the call in Joseph Campbell’s Hero’s Journey.

Dario Amodei has spent the last few years desperately trying to convince the public he is Maximus refusing the call. He would have you believe he built Anthropic reluctantly, to take responsibility for a dangerous technology everyone else would misuse.

For what it’s worth, I think he’s sincere. But sincerity is beside the point, because it doesn’t reconcile with what he’s built.

Over the past year Anthropic has assembled one of the most ambitious commercial machines in technology, from the $100 million Claude Partner Network and Project Glasswing in cybersecurity to an enterprise services joint venture with Blackstone and Goldman and the $150 million Claude Corps placing fellows inside nonprofits.

He’s built a universe aimed at embedding Claude into the economy, cybersecurity, and philanthropy.

As I argued in Models Aren’t Moats, value does not collect in the model. It collects in distribution and lock-in, which is what this machine is built to manufacture.

This is the part that smacks of CS Lewis’ “benevolent tyranny”:

“Of all tyrannies a tyranny sincerely exercised for the good of its victims may be the most oppressive…those who torment us for our own good will torment us without end for they do so with the approval of their own conscience.”

The problem is Anthropic is trying to "thread the needle," casting itself as a self-aware, safety-conscious frontier lab at the same time as it pursues one of the most ambitious for-profit commercial endeavors ever attempted. The market it competes in will, sooner or later, force the very compromise the mission was meant to prevent.

Long Live Open Technology

The case for open technology is age-old, time-honored, and the record is clear.

As we wrote last April, modern computing was built on open foundations, from UNIX and Linux to Python and Git, and today’s giants owe their scale to freely accessible, permissionless software.

Open technology is founded upon principles of liberty, freedom, and basic human expression. A technology controlled by someone else (an individual, a corporation, or a government) can be changed, monitored, priced, or withdrawn at their discretion, and the more it matters, the more that power is worth.

Open technology is no more or less than a simple and effective counterbalance to power: a natural response to forces that would upset an equilibrium.

This is not only the open-source faithful talking. Bruce Schneier, writing the same week, reaches an AI public option as the answer: fund open harnesses that balance capability against safety, and open models whose provenance is public.

AI is a general-purpose technology, and whoever controls it stands to gain enormously, which is why the fight to control it is on. Anthropic spent years trying to establish itself as the lab everyone could trust. In mishandling the public release of Fable, it forfeited part of that credibility and ushered in the era of “Government-controlled AI” in one fell swoop.

The response was fast and collective, the way an immune system meets a virus: the people who care about AI saw a power grab and moved against it.

Power gathers wherever it is allowed to. Open technology is what checks it, not because it is virtuous but because it cannot be taken back once it is out. That is why it endures, and why it will outlast whoever is holding the switch this week.

At 5:21 in the evening on Friday June 12 2026, the US government sent Anthropic a letter. Citing national security and export-control authority, it ordered the company to suspend all access to its two newest models, Fable 5 and Mythos 5, for every foreign national in the world, including Anthropic’s own foreign-national employees. To comply, Anthropic had to switch both models off for every customer, everywhere. By its own account, the company learned the directive existed and had to start pulling the models down inside the same evening.

Anthropic published a statement and posted it the same night. The operative passage:

We are complying with the government’s legal directive and are removing access to Fable 5 and Mythos 5 for all users. However, we disagree that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people. If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.

As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles.

The stated reason was a jailbreak. The government had seen a technique for getting past Fable’s safeguards, and according to Anthropic the technique amounted to asking the model to read a codebase and point out its flaws, a capability the company says is already available in other deployed models including OpenAI’s GPT-5.5 and used every day by the people who defend software for a living. Anthropic is complying, and disputing, at the same time. It says recalling a model deployed to hundreds of millions of people over a narrow finding is disproportionate, and that if the standard were applied evenly it would halt new model releases across the entire industry.

They are right about that. I want to be precise that they are right, because what follows is not a complaint about Anthropic. It is a description of a machine, captured in the moment it was switched on.

I have been making this argument since at least April but I’m honestly surprised it came this fast and in this manner. In Artificial Good Enough Intelligence (April 28) I argued the frontier was going closed, that the best models would increasingly be withheld, restricted, or selectively exposed. In AI Waves #6 (April 23) I wrote that the closed labs had begun to gate capability by trust level, not just price. What I had wrong was the hand on the gate. I took it to be the labs, withholding the top tier as a business decision. Friday the state reached in and took the gate for itself. The off switch was always the risk. It just changed hands.

The Reversal Test

On Wednesday, in Policy on the AI Exponential, Dario Amodei proposed that frontier models pass mandatory third-party testing and that the government hold the power to block or reverse a release that fails. He asked for this explicitly, as the responsible path, with the caveat that the power be exercised through a process that is transparent, fair, clear, and grounded in technical facts.

On Friday, a release was reversed. Against him. Through a process that, by his own account of it, was none of those four things. The letter arrived without specific technical detail, gave one evening to comply, and rested on a finding the company considers trivial and widely available elsewhere.

The gap between Wednesday and Friday is forty-eight hours, and it contains the whole argument. The power to reverse a deployment is not safe in proportion to how carefully it is designed. It is dangerous in proportion to the fact that it exists. Anthropic asked for an instrument and was handed the sharp end of it two days later, which is the oldest lesson in the book about building machinery and assuming you will be the one holding it.

You Cannot Recall Open Weights

Here is the part that matters past this weekend. The reason a directive could darken Fable and Mythos in an evening is that they are closed. They run where the company can reach them, which means they run where the government can reach the company. A single letter to a single legal department took two models away from hundreds of millions of users at once.

There is no equivalent letter for open weights. Once a model’s weights are public, they sit on tens of thousands of machines in dozens of jurisdictions, and there is no address to send the order to. You can regulate what people do with them, you can prosecute misuse, but you cannot recall them. The off switch does not exist, because there is no single hand for it to sit in.

This is the entire case for open models, and it has nothing to do with whether open or closed is safer in the ordinary sense, the sense of refusing a dangerous request. It is a point about who holds the switch. A closed model concentrates capability in a way that is efficient, governable, and revocable. Friday was a demonstration that revocable means revocable by someone other than you, for reasons you may consider trivial, on a timeline you do not control. Open weights are messier, less governable, and harder to make safe in the ordinary sense. They are also the only architecture with no switch to seize. You trade one kind of safety for another, and Friday clarified the price of each.

The Counterweight

This is why the open layer is not an ideological preference. It is a structural one. A world with only closed frontier models is a world with a small number of off switches, held by a small number of parties, that turn out to be live. The counterweight to that is not a better-behaved lab or a wiser regulator, both of which require the institution to stay good forever. The counterweight is infrastructure that does not have an off switch in the first place: weights anyone can run, compute anyone can rent, verification anyone can check. Capability that no letter can recall.

I spent two essays arguing that the threat worth pricing was the concentration of control, not the awakening of the machine. I did not expect the argument to be settled this fast, or this literally, or at Anthropic’s expense rather than in its favor. Friday was not a story about a dangerous model. Every account, including the government’s, agrees the model was not the danger. It was a story about a switch, and the discovery that it works.

p.s. I’m printing these t-shirts. If you managed to trip the safety limits in the last 72 hours you get one. Reply here and I’ll send you one or feel free to copy the idea.

This past Friday President Trump announced he was considering having the government take equity stakes in the largest AI companies, with the proceeds routed back to the public as dividends. No stranger to controversy, breaking tradition, or pursuing strategies historically considered anathema, the President has already directed the government to purchase about ten percent of Intel, ostensibly on the public’s behalf.

Never one to sit out a discussion about wealth redistribution, even Bernie Sanders has proposed a bill for fifty percent federal ownership of the AI megalabs in question. It is not every day that Trump and Sanders reach for the same lever.

In the hallowed words of Oscar Wilde, life imitates art far more than art imitates life. These forecasts have become the genre through which we read the present, and we’ve stopped asking which kind of film we’re in.

AI 2027 and Situational Awareness

Nationalizing frontier AI labs was predicted more than a year ago by a scenario titled AI 2027, a detailed, month-by-month narrative forecast of how AI could progress from today’s agents to superhuman AI and superintelligence by around 2027. Leopold Aschenbrenner’s Situational Awareness, published in 2024, also predicted nationalization, claiming that AI would eventually become a matter of national security.

Both sets of authors have been remarkably successful in their predictions thus far, and they deserve credit for the quality of their research. The AI 2027 authors published their own self-evaluation, and other independent websites now exist that attempt to track its predictions one by one. The capability and safety predictions are the ones tracking closest, several of them early, which is exactly the point: the question is not whether these systems keep getting more capable, it is what that capability turns into.

Aschenbrenner was recently profiled in the Wall Street Journal and boasts impressive returns on investments made on the convictions in his manifesto.

But how do we go from powerful technology to nationalization in a country so wedded to free-market capitalism? How is it that both Republicans and Democrats publicly agree it is the right course of action in the midst of such an aggressively polarized political environment?

Superabundance or Apocalypse?

Since the launch of ChatGPT there has been a steady drumbeat of headlines, proclamations, essays, books, and podcasts all pointing to only one of two hyperbolic outcomes for AI: superabundance or apocalypse.

If the headlines are to be believed, we are either headed for paradise or AI kills us all.

Presenting technological revolutions as the difference between life and death is common. Everyone is interested in the future, and clear, simple outcomes are easy to communicate and easy to understand.

Simple, extraordinary, hyperbolic outcomes are also evidently a fantastic fundraising strategy, because it is hard to raise a hundred billion dollars for a merely useful tool.

Even before founding Anthropic, while still at OpenAI, Dario Amodei was wary of releasing GPT-2, judging it too powerful to put out. He has since written the paradise version in Machines of Loving Grace and the abyss version nearly everywhere else. Demis Hassabis, the field’s designated good guy, keeps sounding the alarm too.

The Messy Middle

But clear and simple outcomes rarely materialize, and grappling with the more probable messy middle means entertaining complexity, nuance, and uncertainty. That takes unusual technical literacy and several rare personality traits most people do not have. Much of this technology is genuinely unprecedented, the frontier is moving, development is accelerating, and diffusion is uneven. So very few people have the tools to evaluate a hyperbolic headline, which is exactly what makes the headlines work.

I made this point last October. We are confidently building toward a future we cannot picture. Eric Schmidt and his peers forecast crisply to 2030, maybe 2035, then simply stop, because no one has a framework for the other side. The honest position, the one that does not trend, is that nobody knows.

But nobody knows is not the same as no forecast. The honest forecast is just a less dramatic one, closer to a couple of points of GDP than to heaven or extinction. Declining to pick paradise or apocalypse is not declining to predict. It is declining to predict theatrically.

The set of plausible futures has exploded, the frontier feels almost infinite, and grappling with infinity is how valuations detach from reason and people start invoking God when they mean machine intelligence.

Suspending Disbelief

The primary problem with these frameworks is the massive, albeit subtle, logical leap they require to go from powerful technology to existential threat to humanity.

The leap is a hallmark of science fiction, and this is the scene where the film asks for it. To lose yourself in Battlestar Galactica, Star Trek, Star Wars, Dune, or Ender’s Game, you first agree to stop asking whether the world on screen could exist. The forecasts ask the same of us, and most of us oblige.

To be clear, the fear of runaway adversarial intelligence is rational. We are right to be afraid of runaway rogue intelligence. But it is misdirected, because it rests on an unfounded assumption. Hidden behind all of the rigorous terminology involving FLOPS of compute, orders of magnitude, exponential development curves, and real, observable trends, there is a simple leap: increasingly capable AI begets self-directed AI.

We’re being invited to believe that really, really good software leads to self-directed, autonomous machines that will want to do good things or bad things depending on how well they are aligned with humanity.

At face value the leap is not immediately evident, which is why it is easy to make. Indeed, long-running agents are already effectively self-directed. But they are self-directed in the loose sense we described in Much Ado About Autonomy:

It is true in a loose sense that agents are programs that can act independently. As I wrote a few weeks ago in AI Waves, an agent is a tool that wields its own tools, does its own work, and has the capacity to operate on its own.

Loose autonomy is independence in execution: humans identify objectives and delegate agents to execute against them. It is useful, valuable, and applicable today.

But autonomous agents also implies a stricter, maximalist form of autonomy: an agent originating its own objectives and operating on its own authority.

The strongest version of the other argument does not need the leap at all. It points at a measured trend: the length of a task an agent can finish on its own is doubling every few months, and Anthropic reports its own models now write most of its code. Extend the line, the argument goes, and the human supervisor quietly disappears. But a longer leash is not a different animal. A model that runs unattended for twelve hours is doing delegated work for twelve hours, not choosing its own ends. What is growing is loose autonomy, execution stretched over a longer horizon. Strict autonomy, originating the objective, is a separate claim, and a rising horizon is not evidence for it.

It’s possible that agents develop strict autonomy, but it is not probable, and it is certainly not inevitable. The line between the two is not clean. A delegated goal spawns its own sub-goals, and a system told to improve itself is, in a narrow sense, choosing what the next system optimizes for. Whether that gradient produces an adversary or simply a more capable instrument is the real question, and it deserves its own essay. The leap is not proven either way. It is just routinely assumed in one direction.

AI is getting extremely good, better than humans, at programming, but despite all of the colossal investment and trillion-dollar valuations, good programming does not necessarily generalize to powerfully adversarial self-directed agents. Great programming will have massive implications for many things including science, medicine, math, and software, all of which will greatly impact society, but it does not have to mean that machines become self-directing, and it does not have to follow that AI is an existential threat.

Asymmetric Information

Artificial intelligence is distinct from other transformative technologies in one respect: it empowers both individuals and nation states.

What AI spokespeople claim when they present the situation is that incredible technology may one day become more powerful than humans. That was true, for example, of nuclear weapons.

Individuals cannot wield nuclear weapons, however, whereas they can use AI to help develop bioweapons, which is why OpenAI and Anthropic, alongside a roster of scientists, signed a letter this month urging lawmakers to tighten screening of the synthetic DNA sequences that could be used to build them.

Asymmetric warfare is a good analogue too. Whereas much of nation-state military spending is focused on increasingly powerful and expensive weapons development, low-cost, unmanned drones powered by advanced AI and deployed by small groups have completely and irreversibly redefined modern warfare, as demonstrated in Ukraine and the Middle East.

The Individual Matters Most

Which brings us back to the government. Perhaps governments are more concerned with nationalizing AI so that they are equipped to protect against all enemies, individual or collective. Concentrated, already-dangerous capability exists, and governments have decided it is already strategically decisive.

Investing in Intel was ordinary industrial policy, a strategy to own a piece of the infrastructure buildout. Reaching for the model companies themselves, on the other hand, is more direct-to-consumer: whether the models become autonomous, conscious, or rogue does not matter.

We’ve been led to believe that the primary threat involved in AI is rogue intelligence, Terminator-style. But it is much more likely to be rogue humans who learn to wield AI in novel, dangerous, and unpredictable ways.

The difference is not cosmetic. If the danger is a rogue machine, the response is to control the model: align it, cap it, build the kill switch, perhaps own the weights outright. If the danger is a rogue human holding the machine, the response is to control access and the chokepoints: who can point the tool, at what, and through which supplier of compute or synthetic DNA. Misdirected control is not a smaller problem than the one we are bracing for. It is the same problem aimed at the wrong target.

By the same token, AI-enhanced operators with good intentions are now equipped to do things that were not possible before. As an investor and an optimist, I am focused on this last reality, but it is just as important not to underwrite paradise as a business plan. Many AI-skeptics believe in the technology and doubt the valuation. It is possible that OpenAI, Anthropic, and the rest build genuinely transformational technology without becoming enduring businesses. The dot-com build-out was real, most of the companies that ran it did not survive it, and the fiber they laid is still in the ground. Transformational and enduring are different words.

If we eschew the hyperbole, AI is a transformative technology. Period. Through that prism you can see where value actually accrues, and the dark fiber is the warning. The rails get overbuilt, so owning raw capacity is not the same as capturing value. What lasts is the scarce layer: the work of improving the models, the verification that makes heterogeneous capacity trustworthy, and the operators who wield it all best. The boom always lays down too much supply and too little of that.

We are surrounded by narratives that jump from capability to civilizational destiny in a single bound. Life imitates art, but we have been bracing for the wrong genre. The blockbuster ending is the part that needs the suspension of disbelief. The story actually unfolding is the docudrama: the people this technology makes powerful, the people it makes dangerous, and the fight, already starting, over which is which.

As Karpathy wrote, agents actually crossed the chasm in ~November/December of 2025.

What’s more? As Ethan Mollick notes, the AI labs have generally been right. It is extremely tempting to judge the labs and their predictions almost instantly (see Dario’s prediction that AI will render knowledge work obsolete in the near future…). It doesn’t help that enormous amounts of money depend on these judgments and their accuracy, either… but maybe we should *try* thinking a bit longer-term.

OpenClaw

In any case, by all accounts, 2026 will, in fact, be the year of the agents, and Exhibit A is OpenClaw.

Much has already been written about OpenClaw. I will not be contributing another explainer. For a comprehensive overview, please see here.

If you aren’t already aware, OpenClaw is an open-source, autonomous agent created by Peter Steinberger that runs locally on your machine, connects to your messaging platforms – WhatsApp, Slack, Telegram, Discord, iMessage, Signal, Teams, etc – and operates as a persistent daemon: always on, always listening, always acting.

Apparently, Steinberger was personally losing $10,000 to $20,000 per month to keep it running. In any case, it’s no exaggeration to say that it has ROCKED the AI world, in every possible way. Reactions range from measured enthusiasm because agents finally demonstrated utility to the typical breathless excitement over technology deemed “life changing.”

Although creating it is an immense achievement worth recognizing, the core technology isn’t actually revolutionary. It’s a well-designed wrapper connecting an LLM to local tools and services. What made it take off is that it put familiar pieces together in a way that felt, for the first time, like having a genuine digital assistant rather than a chatbot you open and close. It’s what everyone was hoping for in 2025.

As such, traction and usage exploded. To speak of product-market-fit is a dramatic understatement: as with nearly every popular AI-native invention, adoption curves don’t resemble anything else we’ve ever seen in history. OpenClaw crossed 100,000 GitHub stars faster than React, TensorFlow, or Kubernetes. The adoption curve is literally vertical.

There is perhaps no better demonstration of its importance in the AI landscape than the end to the beginning of this fascinating development: joining OpenAI. Yupp! Sam hired him within 3 months.

The Problem

As you can imagine, with great power comes great responsibility, and most people don’t care about the responsibility part.

Part of the reason OpenClaw is so effective is that it operates with the full privileges of its host user. That means shell access, file system read/write, OAuth credentials for connected services, and any digital access keys a user is willing to give it. For that reason, users began buying out dedicated Mac Mini computers for a “clean” environment on which to run their instance of this software.

Meredith Whittaker, President of the Signal Foundation, had already articulated the fundamental tension involved with giving agents root access to computers at SXSW last year.

Nevertheless, tens of thousands of people gave an autonomous AI agent root access to their computers.

Source: https://www.wired.com/story/malevolent-ai-agent-openclaw-clawdbot/

VentureBeat put it best: “OpenClaw proves agentic AI works. It also proves your security model doesn’t.” As a quick overview of the security hazards involved 😂 :

• Security researchers found 40,000 exposed OpenClaw instances on the public internet

• BitSight deployed honeypots and saw exploitation attempts within minutes.

• Because SOUL.md – the file defining the agent’s identity – is writable, prompt injection attacks can achieve persistence: an attacker tricks the agent into rewriting its own instructions, creating a backdoor that survives restarts.

• ~7% of the skills available on Clawdhub were found to be malicious – installing stealer malware that harvests browser passwords, crypto wallets, and session cookies

• And much, much more…

CrowdStrike released an enterprise-wide security framework for OpenClaw. Cisco’s security team called it “groundbreaking” from a capability perspective and “an absolute nightmare“ from a security perspective.

The Opportunity

All of which is to say there’s an opportunity here. For what it’s worth, I’ve seen this before. At Sun Microsystems, proprietary infrastructure migrated to commodity hardware and Linux. When that happened, companies like Red Hat built extremely valuable businesses implementing frontier software for enterprises.

In fact, enterprises will pay enormous sums for someone to make free software reliable, secure, and compliant.

Chamath mentioned as much on the All-In podcast a week ago, saying that “on-prem is back:”

“Once you use these tools, it is very difficult for a company to be able to control how their data is used subsequently thereafter… If you’re using a set of agents to act on all that information, all those agent traces are going back to these model builders... When they realize that that is a problem, the enterprise will have to decide, do I just give up and keep running all of this stuff in the cloud in a shared experience or do I bear the incremental cost of running this stuff in a more coordinated manner that I control on prem?”

Supporting his claim, Chamath pointed to Judge Jed Rakoff of the Southern District of New York’s ruling in United States v. Heppner that said “documents generated through a consumer version of Anthropic’s Claude AI were not protected by the attorney-client privilege or the work-product doctrine under the circumstances presented.”

There is also data suggesting that 86% of CIOs now plan to move some workloads back on-premise, and Lenovo’s 2026 TCO analysis found on-premise AI inference achieves breakeven in under four months versus cloud.

So is there an AI-native version of Red Hat? (Reach out if you’re building this!) We think it may involve something like Palantir’s “Forward Deployed Engineer” model. For a brilliant breakdown of the FDE, see Marty Cagan’s piece here. There are a few highlights:

What makes Palantir’s model work: FDEs build prototypes on top of Palantir’s platform. A platform product team then generalizes what FDEs learn across clients into reusable capabilities. This prevents bespoke sprawl while keeping the deep customer intimacy.

The broader principle: send your best engineers to sit with multiple customers, see the problem space firsthand, and discover solutions that serve real needs across them. This is customer discovery at its most effective.

It goes without saying that there’s lots of data demonstrating that this model works, too (see $PLTR’s market cap, their margins, and their growth). Palantir’s FDEs embed directly with clients to configure and deploy their own software against real operational problems, but it’s no stretch to imagine some cracked team of devs organizing to form a modern form of elite AI-consulting configuring bespoke on-prem instances of the latest AI. In fact, Cagan breaks down how and why traditional consulting models (specifically Accenture) break down when faced with AI.

As such, the AI-native, Red-Hat-meets-Palantir model could become especially valuable because understanding the customer environment in the context of AI is more complex and critical than ever.

To be clear, that model is as follows:

1. Red Hat for the software layer

   1. Open-source, or commodity agent frameworks as the core, enterprise subscriptions for security, compliance, and support

2. Palantir (the FDE framework) for deployment

   1. Embedded engineers who configure agents against specific data, workflows, and regulatory constraints

Cloud isn’t dying, either. What’s happening is selective repatriation: enterprises are moving their most sensitive, high-utilization AI workloads on-premise while keeping burst and experimental workloads in the cloud. But for many organizations (including nations!), control is all that matters. As autonomous agents handle more and more proprietary data, the trend is clear.

Nazaré’s Position

The industry’s consensus remains scale – more compute, larger models, bigger data centers, etc. But other problems exist, the investment space is less crowded, and returns are therefore (much) more attractive. The “who, what, where, why, and how” for AI is critically important, especially within real organizations.

OpenClaw means that 2026 will be the year of agents. We know that models like those of Red Hat and Palantir for enterprise open-source and embedded deployment are feasible and lucrative.

We’ve believed for some time now that, among many other opportunities in AI, the company that configures, deploys, secures, and maintains autonomous AI agents on sovereign infrastructure will build one of the most important businesses of the next cycle.

If you’re working on that, we’d love to hear from you.

7@nazare.io

nazare.io

Welcome to another edition of Robot Wave, where I explore trends in AI.

This has been a big week for AI. Not only is 2025 shaping up to be the year of the agents, but it may also be the year that we Make AI Cheap Again (MACHA)?

Late last year a Chinese company Deepseek released a new model which claimed to use significantly less compute to train than similar models from OpenAI and Anthropic. The latest release on 20th Jan 2025 claims to have similar results to ChatGPT o1 models, again with significantly smaller compute for training. So what’s going on?

Let’s first rewind a bit. I first introduced the concept of MACHA a year ago. What struck me at the time was how the entire AI industry was moving in one very similar direction - just add more compute, more data and of course more $ to the same architecture. The concept of innovation on the model (an area I was trained in through my research in the 90s) was mostly ignored in favor of more compute and more data (as an side note, my PhD was focused on scaling neural network training using Mixtures of Experts). The concept of “scaling law”, where to get better performace, more data and more weights / compute are added to a system has resulted in fantastic performance such as OpenAI and Anthropic but also fantastical investments ($1bn seed round for Ilya’s SSI company, XAI 100,000 H100 cluster etc).

“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation. There were many examples of AI researchers' belated learning of this bitter lesson, and it is instructive to review some of the most prominent.”

Rich Sutton

The entire enterprise seems akin to a modern data AI Tower of Babel - a literal race to the top to meet God (AGI).

““Come, let us build ourselves a city and a tower with its top in the heavens, and let us make a name for ourselves; otherwise we shall be scattered abroad upon the face of the whole earth." The Lord came down to see the city and the tower, which mortals had built. And the Lord said, "Look, they are one people, and they have all one language, and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them.”

— Genesis 11:1–9

Commodity hardware and software eats the world

Back in the year 2000, after finishing 3 years of consulting work at NASA Ames and exploring dotcom ideas, I joined a small company called Infrasearch which was started by some of the early developers of Gnutella, an open source file sharing tool. Infrasearch developed search technology based on the ideas of Gnutella and new XMLRPC technologies. We tried to raised venture funding and even though our seed investors included Marc Andreessen, we ultimately were acquired by Sun Microsystems in 2001.

At Sun I initially ran the P2P research team, JXTA and then started a new project focused on archival storage using clusters of commodity AMD servers and commodity hard drives connected using self repairing and managing software.

What’s the connection to today you may ask? Well at the time Sun made the majority of its money from selling large expensive servers running closed source operating system software. Sun made most of its money from a small number of customers and had been extremely successful in the dotcom boom along with Oracle and EMC. During the dotcom boom I had been working on open source web and operating systems such as linux, mysql and php running on commodity servers. These approaches were often eschewed by investors and the startup MBA tourists who ran many of the dotcoms. In the end the scrappy commodity hardware and software designs were the winners. The impetus for the change, however, was a financial shock in the form of the dotcom bubble bursting. This moment led IT managers to question the cost of the Sun and other machines and instead try machines from Dell or other players using Linux and the open source web stacks. A similar event occurred after the 2008 GFC leading to a move to cloud computing away from self hosted infrastructure.

Deepseek

I believe a similar event is happening now. Deepseek has blown open the door to a Pandora’s box of open source AI technologies which was first opened by the “leak” by Meta of Llama, the open source LLM in 2023.

Our investment in Vast tracked this change last year. Vast provides mostly consumer grade Nvidia chips in a GPU rental market. Vast is growing rapidly and while most applications are focused on inference, new technologies such as Deepseek and also distributed training systems such as Nous Research, PrimeIntellect (Nazare portco) and Gensyn may end up disrupting the current training paradigm. This well written piece presents the hardware and software case against Nvidia (more), whilst Musk and others are questioning whether Deepseek was really trained using so few GPUs. The code is open source and some researchers are already reproducing the results.

Deepseek combines 2 things we are already familiar with (see this review also)

• Mixtures of Experts (MOE) architectures in which sections of the billions of weights are turned on and off during training and inference for efficiency,

• Quantization: using KV cache compression to make the training and inference more efficient.

The final innovation is to combine these first 2 optimizations with the Chain Of Thought reasoning of OpenAI o1 and a Reinforcement Learning approach. This final optimization seems to be the technique that results in the reported dramatic ($5.6m) reduction in cost of training (compare this to the 100k H100s of XAI).

Naturally people are skeptical, and many researchers are attempting to reproduce the results. Some people are suggesting its a Chinese psyop and China has possession of H100s, thus violating the US sanctions.

I am in the camp that this is a real innovation not a fake one. However it plays out, the genie is out of the bottle that the perception is that we can do more with less - we can train and execute large models using less hardware, potentially commodity hardware and without the incredible high cost of investment in hardware infrastructure.

What’s next?

Do the massive clusters of Nvidia’s most powerful chips now become irrelevant? Not necessarily. Even as the LLM market changes dramatically with less reliance on compute, the quests for ASI and AGI – humanity’s Babylonian pursuit of finding “God” – will likely still be dominated by throwing maximal resources at the challenge. Over time, however, I expect that software improvements will keep strong moats from developing in these areas just as we’re beginning to see happen with LLMs. This is now a brave new world. What was taken for granted is being questioned, prior assumptions are being challenged, and new possibilities abound.

Till next time,

Steven

Dr. Steven Waterhouse

Founder and General Partner,

Nazare Ventures

X, Linkedin