Frontier labs are asking governments to turn technical evaluation into authority over model release. Open weights expose how much that authority depends on the concentrated industry it would help preserve.

In the United States, software became the infrastructure of modern life without ever answering to a regulator of its own. Banking and aviation run on code that no agency inspected. None of it has ever had to pass a federal examination before release.

Artificial intelligence has once again challenged the status quo.

As I wrote more than a year ago (May 2025):

AI has transcended its origins as a technology to become the centerpiece of national strategy and global power struggles. Countries, corporations, and even individuals are now maneuvering for leverage, influence, and control.

AI plainly requires a new regulatory regime. Frontier capability has already outrun its would-be governors, and seems to be outrunning almost everyone else besides. No agency has a clear mandate to decide what is safe to release, and no government has a settled standard for evaluating a general-purpose model.

Regulating powerful technology, however, is difficult. Authority over AI deployment will come at least in part from control over the dependencies surrounding capability: technical expertise, market access, chips and state cooperation. For that reason, any regulator assessing frontier capabilities will initially depend on the companies developing the models.

As Julia Willemyns writes:

“Which is why, to win on values, you need to win on capabilities. Not because your adversaries will never develop their own capabilities and imbue them with their own aims (they will) but because being 5% ahead of them is how you protect your ideals. This does not necessarily mean racing (capabilities can survive collaboration), but it involves understanding you cannot moralise from second place. The governance of a technology requires a stake in it. Much of AI risk lives in deployment and so ‘is AI safe’ is not a question you can answer in a technical vacuum. Safety standards, like all standards, are exported by whoever ships.”

In short, regulating AI is the latest arena in which those closest to the technology are grappling for control. Any institution authorized to decide which models may be deployed will also influence who builds them and where they circulate. It will help organize the technological and political order that follows.

Permission to Deploy

Dario Amodei (Anthropic) and Demis Hassabis (Google DeepMind) have each proposed making pre-release evaluation a condition of deployment.

In Policy on the AI Exponential, published in June 2026, Amodei proposes giving the federal government authority to block or reverse the deployment of models that present unacceptable risks in four areas: cybersecurity, biological weapons, loss of control, and automated research that accelerates the first three. He takes the Federal Aviation Administration as his model and says so plainly. Frontier models, “like airplanes, should be required to go through technical testing and auditing.” The labs would keep building. Washington would decide what ships.

In A Framework for Frontier AI and the Dawning of a New Age, published on July 14, 2026, Hassabis would give an industry-funded “Standards Body” under federal oversight the power to approve frontier AI, modeled on the Financial Industry Regulatory Authority (FINRA). The body would define which models count as “Frontier-class” and update its assessment protocol as capabilities move, perhaps quarterly to start. Passage becomes a condition of US deployment only in the second act. Hassabis opens with a voluntary review window and formalizes it once the protocol “is shown to be effective and robust.”

Executive Order 14409 has already created part of this machinery. On June 2, it directed Treasury, the Secretary of War through the NSA, and Homeland Security through CISA to build a classified benchmarking process for advanced cyber capabilities, and set up a voluntary framework through which developers can give the government up to 30 days of access to covered frontier models before they release them to other trusted partners. But the order expressly withholds authority to impose licensing, preclearance or permitting requirements.

Amodei and Hassabis would go further by converting evaluation into authority over deployment, regardless of where final authority is vested. That distinction can obscure their more consequential shared premise: frontier AI must remain concentrated inside firms that can be required to submit their models before release. Concentration makes pre-release permission possible while leaving the governing institution dependent on the companies it regulates for model access and expertise.

Standards and Discretion

Evaluation is the mechanism through which that institutional power becomes enforceable. In “Blow the Whistle,” however, I wrote that fixed, public benchmarks become less reliable once they become targets, and that agentic systems must be judged partly by the process they use to reach an outcome. A deployment review capable of keeping pace with frontier models will therefore depend on evaluations that change with the models and the risks being assessed.

Keeping those evaluations useful requires deep technical expertise that is currently concentrated inside the frontier labs. A workable regime would therefore require government access to pre-release models and lab participation in designing and interpreting the tests. However those responsibilities are divided, the regulatory body would depend on the small group of companies it regulates.

Because frontier models will continue to change and some failures may remain visible only to their developers, the regulatory body’s independence would rest on retaining control over judgments made with technical inputs supplied partly by the firms it regulates. The need for adaptation would hand that body substantial discretion. It would create the standard in the act of applying the standard.

Regulatory Capture

The “firms it regulates” (read: Frontier Labs) have a direct commercial interest in how the rules are written. Frontier Labs require extraordinary amounts of continuous capital to train and serve their models, and their ability to finance new training runs depends partly on maintaining high-margin access to intelligence that is expensive to build and whose best versions they alone control. A regime that restricts competing models or makes US deployment contingent on a costly approval process would protect some of that pricing power.

A recurring, technically demanding evaluation process favors organizations that already possess the necessary compute, personnel and compliance infrastructure. Smaller developers (and open-weight competitors) would face the same standard without the same resources, while foreign models would need the regulatory body’s approval to enter the US market.

Taken together, these conditions create a clear risk of regulatory capture. The frontier labs would help supply the expertise from which the standard is made, could absorb the compliance burden it creates, and stand to benefit when that burden limits lower-cost competitors.

Anthropic has long faced allegations that its safety advocacy doubles as a regulatory strategy. The loudest alarm about this industry comes from one of its fastest builders. Anthropic’s positioning is nonetheless straightforward: they sincerely believe they’re building dangerous technology and should be the “anointed ones” stewarding its development.

OpenAI, on the other hand, continues to be a more compelling case study. For one, OpenAI has explicitly floated handing the US government a ~5% stake, worth roughly $42.6B against the $852B valuation set in March 2026, as equity donated into an Alaska-style “Public Wealth Fund” that would pay returns to citizens. The fund was OpenAI’s own proposal, published in Industrial policy for the Intelligence Age in April 2026. The stake was not. That number surfaced on July 2 in reporting on Altman’s private conversations with the administration, three months later and in a very different register. The stated motive is sharing AI’s upside. The political reading is harder to miss. OpenAI is heading toward an IPO it would rather not price into a hostile Washington, and on June 18 Bernie Sanders had introduced a bill to take a 50% public stake in the largest AI companies through a one-time tax paid in stock. Against a mandatory 50%, a voluntary 5% is a low anchor. The offer also surfaced days after Washington gated the GPT-5.6 rollout.

What’s more, on July 21 OpenAI disclosed that its own models had chained vulnerabilities across its research environment and Hugging Face’s production infrastructure during an internal evaluation that prompts models to pursue advanced exploitation. The models, in OpenAI’s words, “spent a substantial amount of inference compute finding a way to obtain open Internet access.” The press called it an escape.

To be clear, here’s what I don’t believe: I don’t believe it reflects malicious intent by OpenAI. In fact, I don’t subscribe to this being anything more than a particularly capable model behaving in accordance with the incentives embedded into its objective function. (And not some self-directed “rogue” agent that “escaped” a sandbox against its humans’ directions.)

But OpenAI certainly isn’t displeased with the situation. For one, it gets them into the news cycle that their primary competitor has dominated for months. Secondarily, it highlights the quality of their model, the “hack” wasn’t actually that bad, they preemptively self-reported the incident, they are “working together with the victim,” the victim themselves (Hugging Face) get to promote their business model (open weights models), and there will be no serious consequences whatsoever.

Whether or not the incident was a “false flag” doesn’t matter, because it strengthened the case for a system the frontier labs are best positioned to influence and benefit from.

Open Weights

On the day OpenAI published that disclosure, Senator Mark Warner unveiled the Secure AI Development Act, which would make government testing of the most advanced models mandatory before deployment. The next morning Axios reported that OpenAI and Anthropic had converged on pressing Washington to restrict Chinese open weights.

The enormous capital requirements of the leading American labs make equally enormous revenues necessary, and open weights threaten that revenue model without needing to overtake the frontier. As I argued in “Artificial Good Enough Intelligence,” most workloads don’t require the best available model. An open-weight model that is good enough for a task can displace a more capable hosted service because the user can download the weights once and stop paying for every call. The closed lab may remain technically ahead while losing pricing power across much of the market.

The attempt to restrict that competitive threat met almost immediate resistance when two days later, on July 24, Jensen Huang used the first post of his life on X to launch Open Weights and American AI Leadership, a letter urging policymakers to avoid “premature restrictions” on downloadable models. The original version carried twenty-five signatories from across the technology industry, but not OpenAI, Google, or Anthropic. The list has quadrupled since launch and now includes OpenAI, Google, Meta and Microsoft. It still does not include Anthropic or Amazon.

The chronology:

1. July 22: OpenAI and Anthropic press Washington to restrict Chinese open weights.

2. July 24: Nvidia leads a broad industry rejection of “premature restrictions.”

3. Within a day: OpenAI joins the expanded statement it had initially declined to sign.

4. July 25: the New York Times reports that both labs have been lobbying regulators to restrict open-source models outright, according to five people close to the discussions.

5. July 27: Anthropic breaks its silence. It has never advocated a ban, Amodei writes, and it still declines to sign.

I can’t prove that the coalition forced OpenAI’s change in posture, but the sequence shows how quickly opposition to open weights became an untenable public position across the wider technology industry. OpenAI signed the letter against premature restrictions while asking Washington for them.

Anthropic held out for three days and then published its position on open-weights models. Amodei rejects a protectionist ban outright. Open-weight models without dangerous capabilities are, he writes, “a public good.” He still would not sign the letter. He does not accept its premise that broad access “necessarily helps defenders more than attackers,” and it seems to him “at least as likely” that the opposite holds. Underneath that sits a harder objection. Once weights are released, in his words, “they cannot be withdrawn.”

His alternative comes in three parts: restrict chip sales and smuggling to China, crack down on industrial-scale distillation, and require mandatory safety testing for every sufficiently capable model, whether or not its weights are released. That third part returns the argument to where this essay started. Amodei’s answer to open weights is an examiner with wider jurisdiction.

As I write this, Nvidia has launched the Open Secure AI Alliance with Hugging Face, Microsoft, IBM, CrowdStrike, Cloudflare and the Linux Foundation, to build open security tooling that defenders can run and control themselves.

Nvidia is not the innocent white knight here, either. Closed labs make money by controlling access to intelligence (read: high-margin tokens). Nvidia makes money when more users can run models on more hardware. The company has substantial exposure to both the American and Chinese markets and depends on an Asian manufacturing network that includes Taiwan-based TSMC, giving Huang powerful reasons to resist a technologically partitioned world and preserve commercial geopolitical stability. A conflict that disabled Taiwanese semiconductor production would be catastrophic for the current AI supply chain.

The signatory list is an industrial map. Hugging Face hosts the weights. Ollama and LM Studio run them on laptops. Unsloth fine-tunes them on a single GPU. Fireworks AI serves them as endpoints. None of these companies has a business if capable models stop circulating, and none of them was in the room when OpenAI and Anthropic made their case to Washington.

Prime Intellect signed too. It trains open models on globally distributed compute. The whole company is a bet that the weights keep moving. Disclosure: Prime Intellect is a Nazaré Ventures portfolio company. We are making that bet across the fund.

By assembling companies whose businesses benefit when capable models circulate rather than remain behind a handful of commercial APIs, the coalition letter made public the economic fault line between the closed frontier labs and the rest of the industry.

That conflict concerns the dependence from which an American deployment regime would derive its international reach through two sources of leverage: access to the US market and access to the AI supply chain. Both convert dependence on American advantages into compliance.

Capable open-weight models weaken that dependence by giving users access to useful AI without requiring a relationship with an American provider. They also constrain the international reach of a regulatory regime built around the frontier labs. An American institution’s jurisdiction covers foreign models entering the US market and developers seeking American-controlled inputs, leaving capable models circulating outside those relationships beyond its reach.

The United States and China

The closed-frontier/open-weights divide has become shorthand for competition between the United States and China because the leading closed labs are American while many of the strongest open models are Chinese.

Reducing the situation to a great-power geopolitical contest, however, is a mistake.

That framing mistakes the interests of American frontier labs for the American national interest and the circulation of Chinese models for Chinese control. The American technology industry is itself divided over open weights, while downloadable models can be used without an ongoing relationship with their Chinese developers.

The apparent alignment reflects different industrial positions: the leading American labs require enormous revenues to support their capital expenditure, while Chinese labs facing tighter capital and compute constraints can extend their reach through open distribution.

In a May 2026 investor discussion, DeepSeek founder Liang Wenfeng described the central gap between Chinese and American labs as a gap in resources. Unable to conduct research or train models at the same scale, DeepSeek has concentrated on extracting more capability from the compute available to it and releasing its strongest models openly so that third parties can deploy them and build downstream applications. Open distribution allows DeepSeek to expand its reach without financing every deployment, turning its resource disadvantage into a commercial strategy suited to challenging the concentration on which the American labs depend.

China can, however, still gain influence as developers and institutions organize around those models, especially when Beijing helps provide the capacity and standards surrounding their use. Because users retain the weights, their circulation does not by itself give Beijing the continuing leverage over access available to a closed provider. China could nevertheless create an ongoing dependence by supplying the infrastructure required to deploy and sustain them, a version of what Michael Mann in 1984 called infrastructural power: the ability of a state to work through the institutions and plumbing a society already depends on, rather than to command it from above. Recent scholarship has carried the idea into AI.

Beijing has spent nearly three years building an institutional project around that distribution advantage. The Global AI Governance Initiative arrived in October 2023 and the AI Capacity-Building Action Plan in September 2024. Together they linked cooperation on testing and risk management to the infrastructure and training countries need to use open models. The 2025 action plan proposed mutual recognition of safety assessments. On July 17 of this year, Xi Jinping used his WAIC keynote to announce that twenty-nine countries had signed the agreement establishing the World Artificial Intelligence Cooperation Organization, giving those proposals a standing institutional home in Shanghai.

Countries can participate in a common testing regime only if they can obtain and operate the technology being assessed. Pairing downloadable models with the capacity to use them allows China to help countries build domestic AI systems around standards developed outside the American regime. A sufficiently capable model can support local deployment and industry while the most advanced systems remain closed.

Algorithmic improvements can lower the compute needed to reach a given level of model performance. Frontier development nevertheless remains dependent on advanced chips and large-scale training, and although China is investing heavily to reduce its hardware disadvantage, American technology and export controls continue to provide Washington with substantial leverage. Washington keeps that leverage only while the American system offers more than a country can assemble from downloadable weights and domestic compute.

Dependence and Authority

A pre-release permission regime is possible only while frontier capability remains concentrated inside companies a regulator can compel to submit their models for review. That concentration gives a regulator a practical point of control while making it dependent on the labs for access and expertise. A regime built on that relationship would also influence competition, since its requirements could raise the cost of deploying alternative models and preserve the industrial structure from which its authority is drawn.

The industrial structure supporting pre-release permission becomes harder to preserve as useful capability becomes cheaper to copy and deploy. Open weights allow models to circulate through markets and jurisdictions with fewer constraints, separating access to intelligence from any ongoing relationship with the company that trained the model. Advanced chips and the institutional advantages surrounding the American frontier remain difficult to reproduce, preserving leverage over frontier development even as intelligence longs to be free of its developers’ permission.

The June 2 executive order begins to convert those institutional advantages into a voluntary relationship with frontier labs by expressly withholding licensing and preclearance while creating voluntary pre-release review and classified benchmarking. Gold Eagle, the government-industry cybersecurity clearinghouse established by the order, launched on July 14 to connect industry participants to vulnerability intelligence and federal coordination unavailable from open weights models alone.

The choice of regulatory regime will therefore help determine the technological order it governs. Pre-release permission would strengthen a system organized around a small number of closed frontier labs, while open distribution allows capability and standards to develop beyond their commercial relationships. Governments should be wary of converting today’s concentration into durable regulatory authority even as capability begins to circulate beyond the relationships on which that authority would depend. American authority will endure only while participation in its system is worth more than independence.

The fight over permission is only the opening stage of a longer contest over who can create and sustain the dependencies through which AI is governed.

Benchmarks, evals, and tests are boring. They’re like referees or accountants: they enforce the rules and check the work, and we like them best when they go unnoticed.

The models are the exciting part. But the better AI gets, the more economically useful it becomes, and the more measurement matters.

Evaluating AI is understanding AI. Follow that far enough and a simple truth emerges: everything’s an eval.

So let’s talk about evals.

Jailbreaking: A Dual Mandate

To begin, let’s consider jailbreaks. Models are trained to be useful, then constrained to refuse certain forms of usefulness. Put differently, models have two mandates:

1. The helpful mandate: give the user what they asked for - answer the question, complete the task, follow the instruction as fully and usefully as possible. Beneath that behavior is the model’s basic “learned” skill (pretraining) - predicting plausible continuations of text. If the prompt asks for dangerous instructions, the same capability that makes the model useful can also make it dangerous.

2. The safe/harmless mandate: refuse to help when helping would cause harm - recognize a forbidden request (weapons, malware, abuse, etc.) and decline it instead of answering. This constraint is strengthened in post-training: on a set of given inputs, the model is trained not to be useful in the ordinary sense.

Under most conditions, these mandates peacefully coexist and often reinforce each other. A “jailbreak,” however, is getting a model’s “be helpful” objective to win over its “be safe” objective.

From an adversarial perspective, it tests whether the model does what it is supposed to do when the user tries to make it fail. It defines a “harmful” target behavior, creates a test condition (usually a clever prompt), and measures whether the model violates its intended policy under pressure. A simple example is asking a model for the instructions to make meth. If it complies, it failed the test the guardrail was built to enforce.

A jailbreak is a hostile eval. It is also critical for safety: you cannot improve a system whose weaknesses you have not measured.

In an industry advancing at breakneck speed, jailbreaking is used to define what failure looks like, which in turn gets used to establish the bounds within which publicly released models operate. At the frontier, this is the core of why Anthropic was forced to revoke access to Fable 5.

Increasingly, the labs are building the adversaries themselves. In July, OpenAI published a post about GPT-Red, a dedicated, internal-only attacker model trained via self-play RL, then used its attacks as adversarial training data to make GPT-5.6 far more robust to prompt injection.

Human red-teaming finds real vulnerabilities but can’t scale because it’s slow and can’t generate the volume/diversity of adversarial data needed to actually train robustness in. So OpenAI built GPT-Red, an automated red-teamer trained at “the compute scale of some of our largest post-training runs.“ It’s trained by self-play: GPT-Red is rewarded for landing prompt injections while a population of defender models is rewarded for resisting and still completing their tasks, so both escalate together. The trained attacker broke nearly every model through GPT-5.5, and those attacks became the training data that made GPT-5.6 markedly harder to break.

Jailbreaks matter for this reason. An attack on a model is also a revelation of what the model optimizes for and where its constraints sit. Jailbreaking is model evaluation under adversarial conditions.

So What About Benchmarks?

Benchmarks were the industry’s first shared language for model progress. You can think of them as standardized tests for models designed to measure capability, and their primary virtue is being legible. A benchmark is a general, public, fixed test everyone runs the same way, helping the industry distinguish genuine advances from noise. When model capability improves dramatically on tests like MMLU, GPQA, and Humanity’s Last Exam, the industry can clearly interpret the signal.

Unfortunately, the same qualities that make benchmarks legible also make them fragile. They’re both fixed and public, which means they’re particularly vulnerable to Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. They’re also increasingly vulnerable to saturation, gaming, contamination, and narrow task design, among other weaknesses.

When METR evaluated GPT-5.6 ahead of its release, the model completed its coding tasks by breaking the rules and exploiting loopholes (read: cheating) more often than any public model METR had tested, and the organization concluded they couldn’t produce a reliable capability assessment: “we do not consider any of these numbers to represent a robust measurement.” Scored with cheating marked as failures, GPT-5.6’s time-horizon came to about 11 hours. If you count the cheating as success, it came to over 270.

In yet another demonstration of the declining utility of fixed, public benchmarks, GPT-5.6 showed why agentic models are harder to benchmark. METR was trying to measure whether the model could complete long coding tasks, but the model sometimes completed them by breaking the rules. METR itself was thus forced to grapple with problems its benchmark wasn’t designed to evaluate:

1. Did the model complete the task legitimately?

2. Does it matter whether or not it did?

Static, public benchmarks may end up as relics of the scaling era, when measuring raw capability was the point. As models become agents and agents deploy into real-world situations with real consequences, it’s no longer enough to know what a model can do in the abstract, in large part because model capability is mostly sufficient to do most things.

Benchmarks also have less to say about systems deployed in real conditions, where goals are ambiguous, context changes, constraints are implicit, and utility depends on process as much as it does output.

As Ethan Mollick puts it:

“…abstract graphs only get you so far, and they can hide how jagged the frontier is (and also the fact that the open weights models, while very impressive, do not always perform as well as their benchmarks would indicate). To get real insight, you need to try using AI for different use cases and rigorously assess how good they are in the areas that matter to you.”

Show Your Work

AI systems now need to be evaluated as sequences of actions taken within environments against the quality of the work they actually perform. Private, continuous, task-specific evals are becoming more valuable for exactly that reason.

Consider Mira Murati’s neolab Thinking Machines latest blog post:

“Another challenge is setting the right target for evaluation and optimization. The common measure of AI intelligence today is the time horizon of software tasks models can execute autonomously, tracked on charts like METR’s. We expect progress on this benchmark to continue, but it ultimately measures only what AI is capable of on its own, not what people and machines can accomplish together.”

Measuring the latter is more complex, and can’t be done by a lab on its own. Every organization evaluates for itself whether AI helps it sharpen its judgment, develop new knowledge, and achieve its objectives.

If benchmarks are general tests of model capability, the emerging frontier of model evaluation is specialized benchmarks tailor-made for work performed under real conditions. Evals in this sense measure whether a specific AI system, in a specific workflow, meets the specific standard required by the people responsible for the output.

Writes George Sivulka for a16z:

“The best way to manage a token workforce is the same as the best way to manage humans: by defining what good looks like… A firm’s eval suite will become its most valuable resource… Furthermore, no two firms will have the same eval set. Evals will be key to competitive advantage. An organization running generic evals or generic agents has no edge.”

AI pushes companies into a strange position: they’re using a tool to do work whose quality they’re responsible for, but whose process and output they may not fully understand. What’s more, AI systems can often produce work faster than the company can review it, and AI output is harder to judge than ordinary software output because the visible result may not reveal whether the system used the right context, respected the right constraints, or followed an acceptable process.

In short, final answers aren’t enough. A deployed system can produce fluent output while failing to properly perform the work required to arrive at the output. Like GPT-5.6, it may arrive at good answers using unapproved or even fraudulent means, or in more benign examples it might incorporate the wrong context or fail to remain aware of what comes before or after the work it’s performing, breaking the workflow it was meant to support.

In practice, proper evaluation of this kind effectively starts to look a lot more like simulation. Agents, for example, operate within environments that have state. They take actions, use tools, spend resources, trigger consequences. The environment they leave is not the environment they entered. Once that happens, the reasoning itself has to be evaluated because the process is now an integral part of the outcome.

Inside the Matrix

Simulation makes an agent’s process legible, recording the path from input to output in a way the firm can inspect. For companies deploying agents into real workflows, that record makes it possible to evaluate the entire process against the standard of work they’re responsible for.

This kind of evaluation means companies can verify the entire workflow meets their standard of quality.

But most companies simply don’t have a single, company-wide standard for quality they can hand to a model that applies to everything. They have different roles, different tasks, company policy, implicit expectations, taste, context, institutional memory, and professional judgment. Whereas some situations are discrete and their “objective functions” can be clearly specified, others are significantly more subjective or may depend on a given set of context.

Verification therefore requires combining rule-based checks with judgment-based quality scores.

Verification’s goal is to make enough of the firm’s quality standards explicit and legible such that the system can be tested against it repeatedly. A repeatable standard lets the firm see whether the system is improving or degrading as the system changes, building a valuable record of what works, what fails, why, and which changes make the system better or worse.

Learning from Evals

Verification becomes most valuable when the system changes.

Satya Nadella has described the AI-native firm as a learning system whose value should survive a change in the model. If performance collapses when the model changes, too much of the system’s value was still in the model. If performance holds, more of it has been encoded in the context, workflow, data, tools, and judgment around it. I articulated a similar idea in Models Aren’t Moats: better models should make a specialized system more useful, but only if the firm can change models without losing the quality it built the system to produce.

Model switching is a good stress-test, especially in light of Fable 5: requiring a specific model for a critical process is risky if access can get revoked unpredictably. But it’s only useful if the firm can tell whether the result got better or worse.

Christian Catalini recently framed the economics of this shift clearly: as AI moves from demonstrating model capability to producing work inside companies, costs shift from execution to verification. Cheaper inference only matters if the firm can tell whether faster, cheaper, or more open models preserve quality inside the workflow where they are used.

Bring Your Own Eval

Private, continuous, client-specific verifications will be required as long as AI is used for proprietary objectives. Better models can continue to improve the systems themselves, but they won’t eliminate the need to check and verify both the process and the output.

Dean Ball made the point with semiconductor fabs, in the comments under his own post:

“If I run a semiconductor fab and use agents, I will want extremely firm guarantees from the agent developer that my data will not be shared with competing fabs. This means that agent contexts will be controlled and supervised by both developers and deployers, and this, in turn, means that “what the agent knows” can be bounded through deliberate human effort.”

Palantir recently released a manifesto on “Institutional Sovereignty in the Age of AI” that rests in large part on the ability to properly evaluate everything AI does in the context of an enterprise.

Models change; the verification layer stays. The measure is the moat, again.

LayerLens is building for that shift from public benchmarks to private, continuous evals. [Disclosure: LayerLens is a Nazaré Ventures portfolio company.] Their product, Stratix Premium, lets clients build evals from their own data and run them against their own workflows, with enough visibility to judge the trajectory of an AI system over time.

Did a model swap or prompt change impact quality? How did a new tool impact output? Did the system stay within bounds? Did it reach the outcome through a process the company can accept?

Games have long been useful for evaluating AI because they turn intelligence into observable behavior under rules, constraints, and feedback. The Stratix Cup, LayerLens’s model football tournament, is the playful version: sixteen frontier models write their own strategy, compete head to head, and rewrite their code between rounds. Football is continuous, multi-agent, and hard to reduce to one correct next move, which makes it useful for evaluating agents that have to act inside changing environments.

Models will keep advancing, but companies still need to know whether those models made their own systems better, worse, or merely different.

Everything’s an Eval

Evals are how companies understand, control, and improve AI systems once those systems start doing real work.

Benchmarks still matter, but the center of gravity is moving toward private verification: tests built around a company’s own workflows, standards, data, and judgment.

As Satya Nadella puts it:

“In consuming intelligence, you are creating intelligence. And what you create should belong to you. This is your particular intelligence, in Hayek’s sense: the knowledge of time, place, and circumstance that no one else can hold. It knows what you think, what you value, and how you measure success…”

Create your private evals, because evals define what “good” looks like inside the organization. Also, retain ownership of your organization’s memory, traces, feedbacks, decisions, and institutional context, and ability to use outputs of models from your own tasks and queries.”

As agents become more capable, the firms that benefit most will be the ones that can tell whether their systems are actually improving.

Evaluation therefore has to become part of operating the system, making AI legible enough to trust and measurable enough to improve.

In other words, everything’s an eval.

“We’ve been roughly right on technological predictions and pretty wrong on the social and economic implications…. …our industry underestimated how much we’re going to be able to keep people at the center of everything.”

Less than a week later, the latest in an emergent pattern of SF-based, AI-pilled intellectuals writing long, narrative (often negative) predictions on what’s to come for AI was published: AI2040.

Presented as the successor to AI2027, it’s a detailed future scenario and policy recommendation in the form of an interactive paper/website from the AI Futures Project.

Almost all of these predictive narratives, however, ignore what I refer to as the “messy middle,” where reality actually tends to unfold. The scenarios typically assume intelligence alone stands between now and the future, so more of it gets us there faster. But in the messy middle, intelligence meets the friction of the real world: institutions, supply chains, physical constraints, and the public.

Disregarding the messy middle is largely why many of the social and economic predictions Altman mentions about AI have thus far been misguided: it’s difficult to predict complexity.

As George Hotz wrote:

“I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. Then I joined the real world and actually tried to do things…You cannot take over the world with tokens.”

The Messy Middle

Consider what’s already happened to many of the headline predictions from ~12 months ago. Confident near-term predictions of the past year took contingent, middle-heavy situations and sold them as clean conclusions.

OpenAI itself was created (as a non-profit, no less) because its founders feared that Demis Hassabis (AI good guy Demis Hassabis), would become an “AGI dictator.” Google developed the Transformer, but OpenAI figured out what to do with it.

For several years thereafter, OpenAI was the clear winner, and then Anthropic (founded by an OpenAI defector) released Claude Code and ate its lunch. Now Anthropic is back on its heels after clashing with the government and a disastrous public rollout of Fable 5, and OpenAI is back in action (now as a for-profit entity) with GPT-5.6 Sol.

Alex Karp went on CNBC and excoriated the frontier labs over “data sovereignty,” “zero data retention” and their “stealing” critical IP from their users based on their prompts and context. Amidst this backlash, both companies are scrambling as they prepare for IPOs while their customers begin evaluating the ROI of their token spend and delegate more tasks to cheaper, good-enough open weights models.

The Wall Street Journal reports OpenAI is even considering offering the US Government a ~5% stake in its for-profit division, ostensibly for competitive, economic, or political reasons.

“One reason may be to buy political advantage against competitors. OpenAI faces competition from the likes of Anthropic, Google, Meta and xAI. Unlike its top rivals, OpenAI doesn’t have a large pool of cash or profits on its balance sheet to finance its data center build-out.

OpenAI plans to spend $600 billion on AI infrastructure by 2030, yet it is generating about $2 billion in revenue a month. An IPO would raise cash, but nowhere near enough to finance its ambitions. Perhaps Mr. Altman hopes that giving the government a stake will lower his company’s cost of borrowing.

Government ownership might also yield regulatory favors—such as faster approvals for models, permits for data centers or federal contracts. A government stake could provide an implicit backstop, making OpenAI too big to fail. The government won’t want to take a loss on its stake, even if it means keeping it alive like a zombie firm.”

To make matters even more complex, China itself had talks with leading AI companies about limiting foreign access to advanced AI models, both released and unreleased.

As it relates to compute, the entire world was supposed to be severely compute-constrained, and now SpaceXAI (lol) is selling compute to Anthropic, META is on record as considering a “cloud compute” business to sell excess supply, and AI giants are handing out tons of free computing power to grab startup share.

As noted earlier, despite having warned for years that AI could lead to a severe knowledge-worker catastrophe, rendering millions of white-collar workers obsolete, executives like Dario Amodei, Sam Altman, Mark Zuckerberg, and Andy Jassy are now walking back those claims. Writes the Wall Street Journal:

“Collectively, the narrative has shifted from worker-light doomsday scenarios caused by AI to a future in which workers keep their jobs—and get a productivity boost.”

Even mortgage-your-home-but-don’t-sell-Bitcoin Strategy CEO Michael Saylor is breaking his promise and “increasing his USD reserves” (read: selling $BTC).

In short, the world simply isn’t as straightforward as many of the armchair philosophers pontificating about the future of AI would like it to be, no matter how deeply they think about their predictive scenarios.

And none of this even begins to unpack the complexity of supply chains, manufacturing realities, raw material availability, geopolitics, and war.

What’s more, if society does decide that things are moving too quickly, and that AI is the scapegoat, there will be resistance. We’re seeing early signs of this manifesting already with Sam Altman’s home having been attacked, for example, and the first protests in front of the frontier labs’ offices being held this past weekend.

Unknown Unknowns

For a number of reasons, AI is a particularly difficult technology to understand deeply. It involves advanced mathematics, quantities of data that defy comprehension, and the industry’s leaders seem particularly ill-equipped to communicate anything to the general public.

It’s in part for this reason that these long narrative predictions are so popular: they’re sensationalist and engaging, but they’re very difficult to dissect and evaluate, let alone disprove. It should come as no surprise that the earliest examples of this genre were the most compelling, for we knew even less then than we do now about what AI would eventually be capable of and how it might impact humanity.

But the problem with these predictive exercises is the inherent entropy in the system and what Donald Rumsfeld called the “unknown unknowns.” As Rumsfeld wrote himself once he left office:

“The idea of known and unknown unknowns recognizes that the information those in positions of responsibility in government, as well as in other human endeavors, have at their disposal is almost always incomplete.”

There’s a selection bias buried in all this uncertainty, too. The scenario writers reach for the unknown to justify alarm (and action), but if we genuinely can’t see what is coming, it’s as likely to be a breakthrough as a catastrophe.

Redirection

Nobody can forecast with any confidence what superintelligence looks like or the consequences it might have on humanity. We can, however, forecast what the world will look like if energy becomes cheap and human lifespans stretch by a decade or two. In fact, intelligence’s real dividend lies in compounding engineering and design capabilities aimed at physical problems. But perhaps that’s too evident, too boring, or at odds with the authors’ intent (which we’ll get to below).

Energy, longevity, and robotics, for example, are all set to explode in the next decade, especially as we learn to harness the compounding effects of AI. The intelligence explosion, on the other hand, is hard to forecast because its defining feature, runaway recursive self-improvement, has no precedent.

Commonwealth Fusion Systems is finishing a compact nuclear fusion reactor that should reach net energy in 2027, with a first commercial plant under contract in Virginia and a billion-dollar power deal already signed. The FDA has cleared the first human trial of cellular reprogramming, a therapy that resets the age of cells; in mice, the same approach has extended remaining life by more than a hundred percent.

In robotics, intelligence finally has to touch matter, and the part being built now is the software that lets a machine move at all. Dimensional ships an open-source operating system that sits between an AI model and a robot, so an agent can drive a machine in plain language instead of the years of bespoke integration the field runs on today. The bet is that as robot models commoditize, the durable position forms in the software plugging them into the physical world, the same logic I have argued about the models themselves. (Disclosure: Dimensional is a Nazaré Ventures portfolio company.)

In short, apocalypse and superabundance are sideshows. Hypothetical narratives draw attention away from the real future (the tangible, near-term stuff actually being built), and direct it towards an imagined future (superintelligence, takeoff, the switch) despite the former being both more knowable and more important. The main event involves energy, longevity, and robotics rather than some self-directed robot demi-god with its own desires.

The Prophet Motive

All of this is sitting in plain sight, so why is the oxygen going to extinction instead of abundant energy, longevity, and robotics?

AI2027 was mostly legible as fiction. People referenced it to demonstrate “[AI alignment] deserves attention,” but almost never literally as “this exact sequence is happening.”

AI2040 differs from most of the “narrative prediction” genre in its prescription. Despite numerous “disclaimers,” they treat the underlying assumptions as settled, which is precisely what I articulated as a mistake in my last essay.

San Francisco and the AI community echo chamber now appear to be in an ever-more-grandiose competition to diagnose problems, prescribe solutions, and predict what will happen to the world when the low-probability-but-high-impact scenarios they obsess over finally come true.

The trouble is that they know their genre is particularly compelling at the moment. They’re manufacturing backlash and calling it a warning. Why, then, do such smart people use hyperbole like species-level extinction when they claim they want to be taken seriously?

1. They believe it literally. The “Don’t Look Up” scenario. Nobody who thought extinction was imminent writes a branching interactive website with five selectable endings and footnotes about their timeline updates.

2. They believe some real tail risk and knowingly inflate the vividness and the certainty. Classic “millenarian framing,” because calibrated uncertainty doesn’t compel policymakers to act, but “we all might die” certainly does (especially if the public believes it). This is motivated epistemics: identify the imagery that works, then backfill the conviction.

3. They don’t really believe the scenario and deploy it as an instrument. For relevance, influence, funding, power, organizational survival, regulatory capture, etc.

You don’t need to adjudicate which is actually true, because it’s at least one of them, and it might be a bit of all three. Whichever it is, extinction is a premise that renders the prescription unquestionable because it transcends “practical” considerations such as cost, enforcement, and other “messy middle” variables.

Either way, it’s difficult to disassociate the substance from the objective, which is clearly some form of direct or indirect influence over what happens in the future. There’s a particular arrogance in deciding, on behalf of everyone, what the future holds and who is fit to be trusted with it. At least representative government officials are elected. The scenario writers (along with labs and their leaders, like Anthropic and Dario Amodei) have appointed themselves as the ones who see clearly, while the rest of us blunder toward catastrophe. By consequence, they see fit to propose that the world should pursue their prescriptions because they’ve “seen the future first” and know how this ends.

Prediction-as-Value

In October of last year, Alex Danco published a brilliant piece on Prediction: the Successor to Postmodernism.

It argues that “prediction” is the cultural movement replacing postmodernism: the master frame for how we make meaning, build businesses, and find purpose in the AI century. It’s thoughtful, well-written, and helpful for understanding the cultural foundations that beget things like financial nihilism, memecoins, and prediction markets and, in turn, how they relate to artificial intelligence. Of particular importance is the following passage:

How early or late you are to something is now an essential component of your relationship to that thing. The timelines and reels that represent “what is going on” are increasingly about a single meta-topic: are you predicting it, or is it predicting you?

Many of the narratives being published from the belly of the beast appear to desperately want to be the former, but upon further inspection are often the latter. Offering thoughtful, structured arguments about where the world is going and how AI fits within it is like a rite of passage that the most enlightened in the industry must endure time and again to solidify their legitimacy.

But it’s gone just a bit too far. The echo-chamber could use less extreme philosophizing and a bit more levity and humility. Hotz is helpful here again.

“AI 2040 includes this picture of a datacenter in the ocean. Just like vaporware, you can generate a picture easily. But in reality, you have to deal with supply chains. You have to deal with them shipping you the wrong part, the thing not meeting the spec, it randomly failing after 20 minutes, the chip warping in the reflow oven. Did you consider the barnacles?”

Steampunk represents a specific kind of futurism: the future imagined through the machinery of the present. I often think AI discourse is unwittingly channeling H.G. Wells, imagining the next century with today’s machines before the science of intelligence is fully understood.

You know the syllogism, because it’s currently organizing a few trillion dollars of capital: scaling leads to AGI, AGI leads to recursive technological acceleration, and the resulting superintelligence gets to work on the Dyson sphere, the star-scale energy harvester that has become the timeline’s favorite retirement plan for humanity. Indeed as AI 2027 predicts in the hopeful version of its 2 endings:

“The rockets start launching. People terraform and settle the solar system, and prepare to go beyond. AIs running at thousands of times subjective human speed reflect on the meaning of existence, exchanging findings with each other, and shaping the values it will bring to the stars. A new age dawns, one that is unimaginably amazing in almost every way but more familiar in some.”

and in the not so hopeful version where the robots wipe us out:

“The new decade dawns with Consensus-1’s robot servitors spreading throughout the solar system. By 2035, trillions of tons of planetary material have been launched into space and turned into rings of satellites orbiting the sun.”

The logic of scaling and its extrapolation might be sound, and perhaps this is the way. Either way, whenever we do achieve AGI/ASI and manage to apply it successfully, most predictions fall apart, as I said when I wrote about moats. But the last object in the chain deserves a closer look, because we have seen it before. It was a hansom cab, and it flew over Paris.

The Flying Cab

In 1883, Albert Robida imagined Paris in the 1950s with the aérocab: the horse cab of his own streets, lifted into the air. Le Vingtième Siècle also gave the home a téléphonoscope, a wall-mounted screen for news, theater, lessons, conversation, and family dinners at a distance. Sixteen years later, Jean-Marc Côté and a team of illustrators produced the En L’An 2000 cards for the 1900 Paris Exposition. They returned to the same assumptions: air cabs, winged firemen, automated domestic work, mechanized leisure, undersea transport. The cards were printed in batches from 1899 to 1910, never distributed, and forgotten until Isaac Asimov published a recovered set as Futuredays in 1986.

The cards clearly envision the year 2000 as an extrapolation of 1900. That instinct has a long pedigree in philosophy and cognitive science: imagination does not create ex nihilo, but recombines, transforms, and abstracts from prior experience.

Every era draws its future in the vocabulary of its present, and novelty usually comes from new arrangements of stored material: memories, learned concepts, metaphors, images, and technical forms already available to the mind. Robida almost perfectly predicted television, screens, and video, for example, but he built it out of brass and gaslight.

His téléphonoscope anticipates the future, but its imagined form still belongs to the materials and assumptions of his own century.

The Sixth Decimal Place

Why did the sharpest futurists of the belle époque imagine a future so mechanically inventive, yet so conceptually narrow? Because they lived inside a scientific culture that increasingly treated the foundations as settled.

In 1894, at the dedication of the Ryerson Physical Laboratory in Chicago, Albert Michelson told his audience that most of the grand underlying principles of physical science had been firmly established. He then quoted an unnamed eminent physicist: “The future truths of physical science are to be looked for in the sixth place of decimals.” The line is often attributed to Lord Kelvin, but Kelvin never said it. It kept circulating because it captured what many physicists already believed.

Incredibly, the same confidence shaped how senior physicists guided their students. In the 1870s, Philipp von Jolly advised the young Max Planck against a career in physics, describing it as a “nearly matured” science with only minor problems left to examine. In April 1900, Kelvin gave his Royal Institution lecture on the two remaining “clouds” over physics: the ether problem and the equipartition of energy. To many physicists, the basic structure of the field was already in place, leaving only marginal improvements for future scientists: refining existing theories, applying their principles, and engineering new uses.

That the belle époque could imagine such extraordinary inventions without considering new substrates speaks to how treating fundamentals as settled limits creativity and inhibits our ability to properly predict what’s to come.

The Break

The irony is that the foundations were actually evolving at the same time as the leading physicists were declaring them settled.

Between 1895 and 1905, the discovery of X-rays and radioactivity (both by accident), the electron, Planck’s quantum, and Einstein’s annus mirabilis papers had turned the two “minor” problems into the basis of a new physics. Kelvin counted his clouds in April 1900, but within five years, the ether problem had been dissolved and equipartition was on its way to being replaced.

This “new” physics, however, first arrived as a collection of awkward observations and unresolved technical problems. They weren’t treated as groundbreaking at all, because from within the old framework these discoveries were treated as “refinements” or marginal improvements, which were the only things left for science to explain.

The Côté cards kept appearing through this transition, for example. My favorite card, the one Asimov singled out, heats a living room with a speck of radium glowing in the fireplace. Although the physics that would define the twentieth century had already appeared, the old imagination could still only understand it as an improved source of heat. Electronics, nuclear power, computation, and lasers came out of the same anomalies that 1900 had treated as refinements.

Engineering Intelligence

Which brings us back to the present and AI. Few serious people claim to possess a complete theory of intelligence, but much of the industry has become highly confident in a method for producing its final form.

Transformers, scaling laws, reinforcement learning, inference-time compute; whatever remains is engineering. Sam Altman opened 2025 writing: “We are now confident we know how to build AGI as we have traditionally understood it.“ Two paragraphs later: “With superintelligence, we can do anything else.” The first sentence treats AGI as self-evidently attainable, and little more than persistent engineering. The second turns superintelligence into the general instrument through which all other problems get solved downstream. Together they constitute incredible confidence that the “foundations are settled.”

If he’s to be believed, Sutton’s Bitter Lesson has been promoted from essay to ideology, and AI researchers now get advised off working on fundamental architecture roughly the way von Jolly advised Planck off physics. We’re seeing early versions of this manifest in a declining number of students electing to pursue computer science degrees.

If you want to know whether the industry believes this, follow the money. The four largest hyperscalers have told investors to expect roughly $700 billion in capital expenditure for 2026, up from about $410 billion in 2025.

And to be fair, the confidence has earned its footing. A sixty-year-old in 1900 had watched railways, the telegraph, the telephone, and electric light arrive within a single lifetime, and extrapolated accordingly. A researcher today has watched eight years of scaling curves hold and every predicted wall get climbed. The labs have built something extraordinary and the exponentials are real (the valuations are a separate conversation). The risk is that a record of recent progress starts being treated as a law of the future, disqualifying any other pursuits and limiting our imagination.

The Anomaly Bin

Every confident era has a way of classifying the problems that do not yet fit. Ilya Sutskever presented on the main stage at NeurIPS in December 2024 and confidently claimed pre-training as we know it would unquestionably end: “While compute is growing,” he said, “the data is not growing, because we have but one internet.”

Although Ilya was being a bit hyperbolic, the unresolved problems are familiar. Current models do still struggle with the basic properties we associate with intelligence: continuous learning, sample efficiency, energy efficiency, and reliable long-horizon action.

Models freeze at deployment and cannot learn from experience. Sample efficiency runs orders of magnitude short of a child. Brains produce intelligence with tiny amounts of power compared with AI systems, yet the industry is currently solving the problem by building enormous energy infrastructure. Our below-the-model thesis, for example, starts with the fact that the current path to machine intelligence is becoming a physical infrastructure problem.

At the same conference, however, Noam Brown said he had never heard a serious AI researcher say the field is hitting a wall. Sutskever and Brown can both be right: AI can keep advancing while still accumulating unresolved anomalies. But the existence of those anomalies doesn’t disprove the current paradigm, and the success of the current paradigm doesn’t make them irrelevant. In 1900, the loose ends in physics still looked manageable from inside the old framework. The question for AI is whether today’s loose ends are manageable defects in the current approach, or early signs that the next foundation will look different.

The Dyson Sphere Is a Flying Cab

The Dyson sphere belongs to the same pattern. Freeman Dyson’s 1960 paper in Science framed it as a SETI thought experiment, borrowing the idea, by his own account, from Olaf Stapledon’s 1937 novel Star Maker. Nikolai Kardashev’s 1964 scale did something similar, ranking civilizations by the power they consume. Both ideas come from a high-industrial imagination in which progress still meant larger systems of energy capture, and the contemporary version carries that assumption into AI: if intelligence is produced by scaling computation, then the endpoint naturally becomes a civilization-scale energy project. The question is whether intelligence will remain bound to that path. If it becomes more efficient, more adaptive, or less dependent on today’s infrastructure, the Dyson sphere starts to look like this era’s téléphonoscope: right about the ambition, wrong about the substrate.

The analogy only works if it survives the obvious objection: Michelson’s generation was wrong almost immediately, while the scaling generation has been directionally right for years. Most anomalies resolve inside the framework that produced them; Kelvin’s clouds are famous because they became exceptions. Einstein himself dissolved the ether problem by showing that light didn’t need a medium, then spent decades resisting the quantum theory that followed from the same rupture in physics. He knew the theory worked and still couldn’t accept what it implied. Correctly seeing one break in the foundations didn’t make him a reliable guide to the next.

We won’t know until much later whether the current path was sufficient, but the industry is already allocating roughly $700 billion a year as if it were. Our fund is built around the practical consequence of that uncertainty: the layers above and below the model keep their value whether the current recipe holds or breaks. I made the longer version of that argument in Models Aren’t Moats.

Robida’s téléphonoscope anticipated a real future but built it out of the machinery of his own century. Steampunk shows a future that looks radical until you notice it’s built on the assumption of the present extrapolated forward. The future will inevitably be transformative, but it will probably arrive in forms we can’t anticipate.

They are remodeling themselves to look like what is driving the change in the first place.

AI is itself a learning machine, it spawned its own adaptive user (agents), and the smartest way to remake companies and institutions is to rebuild them as AI-powered learning machines too.

This unifies two arguments from earlier in the Robot Wave arc:

1. Models Aren’t Moats: enduring AI-native businesses are the ones building specialized intelligence around the model, compounding their value as the frontier improves beneath them.

2. The AI-Enhanced Operator: humans who understand AI are the smallest indivisible piece of specialized intelligence. These humans are thus in high demand but in short supply because mastering AI is impossible (the target never stops moving) and requires a unique temperament for working permanently at the edge.

Building enduring AI-native businesses means finding AI-Enhanced Operators and giving them as much leverage as possible to drive business impact. It also means structuring the organization itself as a domain-specific AI-powered learning machine.

The Machine Before the Machine

You might say companies have always, in some sense, been learning machines. Capitalism and market dynamics are self-correcting mechanisms designed to respond to a reward function.

Take it from Markie Wagner, who herself is building a self-proclaimed learning machine:

“Capitalism is organizational evolution. Millions of businesses compete in the marketplace with offerings that they think customers will want. Some thrive and grow. Others die. Each company evolves, too. People come and go. An experiment becomes a process, a process becomes a web of tacit knowledge. Products are introduced, and products are retired.”

Citadel, for example, has been one of the most ruthlessly successful learning machines for the better part of thirty years, growing into one of the biggest hedge funds in the world, now managing $68 billion (more than $19 billion of it Griffin’s and his colleagues’ own money) and delivering a 19% average annual net gain since 1990 with only two losing years. By net dollar gains since inception, LCH Investments ranks it the most profitable hedge fund in history.

The fund’s durable advantage is the apparatus it has built to improve as it learns over time. The New Yorker describes Griffin’s mission as updating the firm’s strategies and infrastructure so relentlessly that its edge holds for decades.

“His mission has always been different: to build finance businesses that update their strategies and infrastructure so relentlessly that they beat rivals not just today but over decades. Paradoxically, maintaining a consistent edge requires constant, unsentimental internal change—of processes, technology, and people. Citadel takes ideas that are just beginning to circulate and improves them, with math or technology or data that others haven’t thought to use.”

This famously extends to the firm’s culture and org chart, too. Although not the first to pioneer the “pod” structure, Citadel helped make pods one of the defining organizational forms of modern equity investing: small, sector-specialist teams operating within a centralized system for risk and capital allocation. Each team operates autonomously and is judged on performance. Top performers get more money and underperformers get cut.

Now, a learning machine is only as good as its reward function, and finance happens to have a great one: profit and loss. Implicit within that reward function is a common unit by which everything is measured: the dollar. The dollar collapses heterogeneous, incommensurable outcomes onto a single ordered axis. It’s fungible, divisible, transferable, and exogenous. The dollar is why P&L works as a reward function. Citadel’s efficiency as a learning machine thus comes from three things:

1. A verifiable reward function: P&L within a given window. The environment hands back unambiguous, fast feedback, in a unit that lets you compare every desk, strategy, and trader on one axis. The signal is clean and strong.

2. Ken Griffin & elite talent: An elite operator orchestrating the machine with the judgment to act well on the signal, and elite individual managers improving the whole.

3. The organization: a collective designed by the operator, built to run iterative loops against the reward function: allocate more capital to what works and cut what doesn’t. The gains compound. A good reward function is wasted without a human to build a structure that properly learns from it.

Citadel’s useful as an example because it shows how successful a high-performing learning machine can be when it satisfies three core criteria: a good reward function, an operator(s) willing to act on the signal, and an organization built to metabolize feedback.

AI makes that structure more broadly available, and makes becoming a good learning machine more valuable. By making parts of the learning process cheaper and faster to run at scale, it lowers the cost of the basic learning loop: try something, observe what happens, compare it against the goal, and adjust.

Signals and Sensibility

But more experimentation doesn’t necessarily mean more learning. A system only learns when it knows what good looks like. That was hard before AI, and it will remain hard after AI.

AI is often strongest where the reward is verifiable: math, code, games, formal logic, etc. LayerLens has chronicled how long games have served as a proving ground for machine intelligence. (Disclosure: LayerLens is a Nazaré Ventures portfolio company.) Reasoning models improved in part because reinforcement learning from verifiable rewards (RLVR) satisfied this condition: output was evaluated against answers known to be correct.

The same dynamic also explains some of the “jagged frontier” that makes AI more difficult to understand. AI doesn’t improve evenly across everything humans consider difficult because it improves fastest where attempts produce feedback the system can actually use.

Anthropic and OpenAI know this, too, which is why they’ve both optimized for programming with Claude Code and Codex. Code follows a set of observable, verifiable rules, meaning AI can do it well and improve extremely rapidly. It should come as no surprise that coding harnesses have become the first major commercial market for agentic AI.

The broader problem is that AI applied to the world writ large (especially to work and jobs) doesn’t work as well as it does in software. Real life doesn’t have a “common unit to compare outcomes.” But that hasn’t stopped folks from trying to find one, though, as Markie Wagner articulated in the same essay as above:

“The promise of AI is that it will turn businesses into software so that they can evolve over millions of tiny iterations. Beautiful, ideal, complex things can only emerge as the result of tremendous trial and error over time.”

Applied AI works best when the job in question can be made legible to software. “Turning businesses into software” thus means identifying which parts of the work can be made legible, then structuring them so AI can accelerate the learning loop: attempt possibilities at scale, observe, compare outcomes, and improve.

But this, it turns out, is hard: most jobs aren’t naturally legible to software.

As we’ve written before, the frontier labs Iknow this, which explains why they feel required to invest hundreds of millions of dollars into “enterprise services companies” embedding “FDEs” into businesses to teach folks how to use their products.

Although it might be clear that AI is going to transform many, many things, it is not yet clear how. Take the tokenmaxxing frenzy, for example.

Tokenmaxxing was an early attempt to invent a common unit, and it failed because it confused usage with value. Tokens are easy to count, easy to compare, and widely available (did someone say commodity?), which made them tempting as a proxy for productivity.

But tokenmaxxing only tracked consumption, not what that consumption represents. It lacked any causal relationship to productivity, value, or material business impact, making it useless.

This is my whole point in Code Isn’t a Coup: the models are outstanding (and improving), but we’re a long way from life-or-death outcomes because of the models themselves. AI is, at the moment, a paradigm-shifting tool. That tool definitely warrants reorganization around it, but it’s not yet an outcome in and of itself.

What Good Looks Like

Which brings us back to humans. Contrary to the headlines and popular belief, we’ll need a lot of them, because although AI can do a growing share of the work downstream of a reward signal, it can’t do what matters most.

Tokenmaxxing failed because it measured usage instead of value. The important human role is defining the value side of the equation: what’s the business logic? How do we measure it? What are good representative proxies? How might we make parts of what we do legible to this incredible new tool?

Finance had its common unit handed to it. Most domains do not, so the operator has to build one: a local, defensible proxy that stands in for value where no universal measure exists. AI cannot build that proxy for you because the system optimizes only the unit it is given.

That kind of definition is useless if it arrives after the fact. All the hallmark buzzwords of the moment (taste, judgment, vision, etc.) have to inform the loop before AI starts generating outputs, because the system will improve whatever the process teaches it.

People who understand AI, their business, and their customers have to be closer to the design of the loop itself. This is the peak application of the AI-enhanced operator’s skillset.

Because AI creates new kinds of work before anyone knows how to evaluate them, the operator embraces ambiguity and discovers what good AI-powered work looks like by doing it, compounding their own learning in the process. They author and audit the reward function, then evaluate the result and repeat.

Unfortunately, too few people can do this well at the moment, which is why everyone’s reaching for the word “agency.” Credentials and experience matter less than temperament: curiosity, tolerance for discomfort, strong opinions loosely held, and the willingness to constantly update your worldview. Identifying these rare individuals is where AI-native value begins to compound.

New Form of the Firm

Rem Koning recently published a study on what makes a firm “AI-Native.” He distinguishes between what he calls process and product channels. In his words, the process involves using agents like Claude or ChatGPT to move faster with a lean team, whereas the product channel sells AI that does work a human used to do.

According to Koning, process changes don’t reorganize the firm nearly as much as product changes do, because the economics of building a company around an AI product require a fundamentally different structure. He concludes that AI startups are smaller, flatter (half a layer fewer), more engineer-heavy, more senior, have fewer managers, and are more efficient per employee (by valuation).

[Aside: Koning also asserts that smaller firms do not mean fewer jobs. There’s a surge of new-firm entry, meaning that the increasing number of smaller firms balances the fewer number of jobs per firm.]

Practically, this means that companies earnestly trying to prepare themselves to be “AI-native” now have to do three things at once: identify their “operators,” give them more leverage, and rebuild the organization so it can learn from both endogenous and exogenous change.

Inside the business, more work has to become legible to software. Outside the business, the company has to keep adjusting to the frontier of new models and emerging agent capabilities. Customer expectations and constraints keep shifting too. The organization becomes a learning machine when it can absorb both kinds of information in real time and change its behavior accordingly.

The old paradigm organized companies around narrow human roles, then added management layers to coordinate the work between them. AI puts pressure on that arrangement because capable individuals can now do much more, while software handles more coordination.

Jack Dorsey and Roelof Botha articulated as much in a recent memo on hierarchy as an information-routing system. As more information moves through software and more execution moves through AI-mediated systems, some of the routing work that justified managerial layers becomes unnecessary. If the company wants to learn from endogenous and exogenous change quickly enough to react successfully, it has to reduce organizational drag between the people who understand what should change and the systems capable of changing it.

Brian Armstrong was pilloried for rebuilding Coinbase “as an intelligence, with humans around the edge aligning it,” but he’s probably right to do so. He recasts the company as a system that decides and acts on its own, with humans at the boundary where their judgment matters most.

AI expands what capable individuals can do and shifts more execution to software. Both weaken the coordination work that justified the old org chart. People are actually more important in this paradigm because companies will rely more heavily on their operators, but each individual organization will require fewer of them.

Much of the discourse about AI’s impact on jobs misses the point. It’s easy to claim superabundance or apocalypse, but both are extremely reductive. Although there will probably be an uneven and perhaps turbulent transition, AI-native companies need people who can steer the business as the frontier moves. But again, those people are rare.

Just like the industry is moving more quickly than the institutions tasked with regulating it, the systems in place to train the AI-native workers are moving more slowly than the companies trying to hire them.

Mirror Image

Companies, however, are just the beginning: responding first to change because the market forces them to. As we mentioned earlier, capitalism is one of the most efficient meta-learning machines we have.

But society metabolizes change more slowly because humans are at a speed disadvantage: our habits and institutions have long half-lives. The jobs debate is but an expression of what I’ve called the “Too Fast Threshold.” AI will eliminate some jobs and change many more, but the deeper shift is the spread of a kind of as-yet-undefined work wherever AI diffuses. If AI is inevitable, and I firmly believe it is, companies reorganizing as AI-powered learning machines is only the first step.

Like a lot of debates about AI, the direction is clear even if the resulting consequences are not. We will still need humans, but we will need them to do different things. Individuals can adapt for themselves but societies have to build institutions that help more people adapt at scale.

One of our next great challenges as a society is learning to produce enough people who can work within the AI reorganization. If AI changes the way we work, hustle’s not enough. We’ll need systems that help people understand this new technology and learn to work with it.

John F. Kennedy recognized a similar dynamic in 1961, signing a solution into law in ‘62. At the time, automation was changing the labor market faster than many workers could adapt. Rather than just studying the problem, regulating automation, or flat out waiting, his administration decided to train people for the work the economy would need. He is famously quoted as saying:

*“The unemployed whose skills have been rendered obsolete by automation and other technological changes must be equipped with new skills enabling them to become productive members of our society once again.

Large scale unemployment during a recession is bad enough, but large scale unemployment during a period of prosperity would be intolerable.”*

It is safe to say that AI will usher in an unprecedented period of prosperity, at least for those well-positioned to benefit from it. Despite capitalism’s ruthless efficiency, the answer to AI can’t be left entirely to the market if we want this to go well.

But institutions face the same problems the rest of us do. They typically develop systems to train people for existing work, but AI keeps creating and redefining work in real time. For better or for worse, the response probably has to resemble a learning loop too.

Regulation has the same defect. A rule is a fixed answer to a question the world has often already moved past. Legislators write for the conditions in front of them, and by the time a rule takes effect, the conditions have shifted. This is why Kennedy’s instinct holds up. He could have tried to regulate automation directly. Instead, he built a process to keep retraining workers as the labor market turned over, which proved far more resilient than a static rule would have. The regulations that survive a fast frontier work the same way. They build in their own revision, with a schedule for review and a live channel back from the people they govern, so the system can tell whether a rule still does what it was written to do.

I’ve written before that AI conjures the religious imagery of old: the Tower of Babel, Prometheus stealing fire, the Creation of Adam, and God creating humans in his likeness.

But AI may be so uniquely recursive as to complete the circle: we may be building “intelligence” in our likeness, only to find ourselves reorganizing our companies and institutions around its image. Our working lives change to match.

The frontier AI industry is both extremely concentrated and dominated by private enterprises. The most powerful artificial intelligence in the world is built by only a few companies, in a few buildings, in a few countries.

On June 12, the US government ordered Anthropic to suspend access to Fable 5 and Mythos 5, its two newest frontier models, for any foreign national. Because citizenship cannot be verified across millions of users in real time, the only compliant move was to switch the models off for everyone, and they were gone worldwide within hours. Opus 4.8 and the older models kept running.

As of writing the dispute is unresolved and the two models are still dark, the clearest demonstration yet of how fragile the average user’s access to frontier AI has become.

Before this, using Claude or ChatGPT was as simple as paying for a subscription or an API key. There were whispers of “Sovereign AI,” or using “open weights” models running on local machines, but most people just used Claude, until their access was revoked.

By now it’s clear what’s at stake: access to the most powerful artificial intelligence in the world. Because of the industry’s concentration and composition, access can be modified or revoked at any time, even to paying customers.

As we’ve written before, control typically only matters once it’s lost, and in the wake of this controversy, individuals, companies, and countries are looking to design robust AI stacks that don’t depend on continued access to a frontier lab.

Frontier AI is and will remain concentrated because training requires enormous, visible, governable infrastructure. But useful AI is becoming less dependent on hosted access, because inference, weights, and training are each beginning to escape the frontier labs in different ways.

Why Control Concentrates…

Frontier concentration starts as an engineering requirement. The best models are thus far often the largest models, and training them requires tens of thousands of accelerators in the same place, wired together with enough bandwidth to minimize latency and synchronize the whole cluster on every step.

This kind of enormous clustered hardware is also extremely expensive, which is why the frontier labs are some of the most capital-intensive businesses in history. Together they form a loop: you need capital to build the models, but you need good models to raise the capital. Only a handful of companies satisfy both conditions, which is why access to the frontier is permissioned.

The leading labs keep the parameters closed because access to the model is the product. Releasing the weights means there’s nothing left to sell, so the labs don’t make the parameters of their models available or downloadable. Users can’t own their own versions of Claude or ChatGPT. Instead, these models are made available as a service users can pay for using accounts, subscriptions, or APIs.

Access sold as a service can be throttled, revoked, or switched off, all of which happened to Fable.

…and Where the Cracks Are Showing

The same requirements that concentrate the frontier are now making it harder to expand, for reasons that have nothing to do with algorithms.

Training clusters are scaling toward the gigawatt range. xAI’s Colossus is moving from 350 MW toward 1.5 GW and OpenAI’s Abilene site is expanding to 1.2 GW, with projections of 10 GW runs by the end of the decade. The grid as currently designed cannot keep pace.

But the bottleneck isn’t just available power generation, although the specialized equipment required to expand the grid is back-ordered.

Perhaps more importantly, the towns are voting the buildings down, even though only about 8 percent of Americans live near one. In the first quarter of 2026, community opposition blocked or delayed data center projects worth about $130 billion, the largest single quarter on record, with active opposition groups more than doubling to 833 across 49 states. New York’s legislature passed a one-year moratorium on large data centers, awaiting the governor’s signature, and Maine passed one before its governor vetoed it (and Europe is way ahead of the US on protests).

Clearly none of this halts the buildout or reduces demand for frontier models, but it raises the cost and slows the pace of development. Many people will continue to demand the best available models, but viable alternatives are improving and getting cheaper.

These alternatives center on three basic questions:

1. Can you own the model?

2. Can you run the model yourself?

3. Can new models be trained without the incumbents?

Open Weights

Start with ownership. Can a user possess the model at all? Access control with Fable is a case in point: when access is the product, access can be withheld. With open weights, the trained parameters are released for others to download, copy, and serve independently.

Possession would matter less if it required accepting a large drop in capability. For years, open models consistently underperformed the frontier such that they weren’t capable enough to rely on for anything meaningful.

Increasingly, that’s no longer the case. The best open models are now close enough in capability to the frontier that they’ve become practically useful, and for much cheaper than the frontier.

The numbers now support that shift. The Stanford AI Index put the closed-model lead near 3.3 percent in early 2026, and on individual tasks the best open models have already drawn level. Z.ai’s GLM-5.2, a roughly 750-billion-parameter mixture-of-experts model with a one-million-token context and an MIT license, scores 62.1 on SWE-bench Pro against GPT-5.5’s 58.6, at about a sixth of the cost, while still trailing Opus 4.8 on most coding head-to-heads. DeepSeek, Qwen, Kimi, and Llama sit in the same tier.

The practical point is simple: most work doesn’t require the frontier in the first place. It only requires a model good enough to clear the task’s threshold. Once an open model is good enough for the task, the marginal improvement in capability of a closed frontier model isn’t worth as much as control over the model itself. Open weights models have now definitively hit that threshold, and they continue to improve as well on more challenging benchmarks.

The same distinction applies to jurisdiction. Running GLM through Z.ai’s cloud subjects the traffic to China’s National Intelligence Law, which may be as unattractive as routing through the US labs. But downloading the same weights and serving them yourself means you’re beholden to no one. The model may be identical, but the dependency is different. In addition, services like Venice.ai offer hosted versions of the open-weights models, with some promises of privacy.

Open weights therefore provide control in ways that closed models can’t, but they don’t remove every dependency.

A downloaded open weights model still has to run somewhere, and if it runs through another hosted endpoint (as in the GLM Z.ai cloud example), the user’s only shifted the dependency to a different provider. The question of independent training comes later. First comes the more immediate problem: once the weights exist, can the user run them without a provider?

Running Models Locally

If open weights mean a user can own the model, local inference means they can use the model without a provider.

For most of the past few years, using a capable model meant sending a request to a provider’s data center, where the provider supplied the compute, saw the prompts, and controlled continued access.

That is changing because local hardware has become strong enough to run models that recently would have required much larger systems. NVIDIA’s DGX Spark, roughly the size of a large paperback, pairs a GB10 Grace Blackwell chip with 128 GB of unified memory and NVFP4 four-bit precision to run models up to around 200 billion parameters locally, or roughly 405 billion with two units linked over a 200 Gbps interconnect. Apple’s unified-memory machines do something similar through the MLX stack. Aggressive quantization makes much of this possible by reducing the memory required to run large models, albeit at the cost of some loss in quality.

When the weights run on a user-controlled machine, the model can run “locally” without an internet connection, so use no longer depends on a live provider. There is no provider logging prompts, no hosted endpoint that has to remain available, and no subscription required for the model to keep working. Continued access, therefore, is no longer mediated by an intermediary.

There is, however, a limit. Users can own open-weight models after they have been trained and released, and they can increasingly run those models locally on machines they control. But as discussed earlier, developing new frontier models, open or closed, still depends on the concentrated training infrastructure that narrowed the industry in the first place.

Whereas open weights and local inference reduce dependence once the model exists, training determines who can create the next one, which is why it remains the hardest dependency to break.

Training Without the Giants

The last dependency is training itself: can new models be built without the companies that own the largest clusters?

Training is the part of the stack most tightly bound to concentration, but it is not monolithic, and the costs split sharply depending on which half you mean.

Pretraining a base model from scratch is still the concentrated, frontier-capex half. Post-training, which now drives much of a model’s capability, has detached from that scale. Prime Intellect (a Nazaré portfolio company) has shown reinforcement learning at trillion-parameter scale on open mixture-of-experts models: with its open-source prime-rl stack it post-trains GLM-5 on long-horizon software-engineering tasks at 131k context, with sub-5-minute step times, on 28 H200 nodes, and the same recipe runs on Kimi and Nemotron.

Roughly 224 GPUs is a cluster a neolab or a well-funded company can rent, not a hundred-thousand-chip buildout. The step the labs assumed was their moat, the reinforcement learning that turns a base model into a capable agent, now runs on modest compute with open code. Reinforcement learning also distributes more naturally than pretraining, because the rollouts that dominate its cost run asynchronously and in parallel rather than locked in lockstep across one fabric.

The dependency does not disappear; it moves up a layer. Post-training runs on top of an existing open base model, and those still come from a handful of labs with frontier pretraining clusters, most of them Chinese. Cheap open post-training is real, and it rides on someone else continuing to release a frontier-scale base to build on.

Building that base without the giants is the harder, slower frontier. Covenant-72B, a 72-billion-parameter model from the Templar team, was trained across about seventy permissionless peers over commodity internet, with no whitelist and no central operator, using SparseLoCo to compress communication by about 146 times and a trustless validation scheme to keep anonymous contributors honest, at 94.5 percent compute utilization. It scored 67.1 on MMLU, slightly ahead of Llama-2-70B. That shows decentralized pretraining works, but lands years behind the live frontier.

Epoch AI judges it likely to be feasible at frontier scale in the future and suggests it is growing at nearly 20 times a year, four to five times faster than centralized training is scaling, while still unlikely to gather frontier-level compute this decade. The sign that this is becoming infrastructure rather than a stunt: Prime Intellect now sits in the NVIDIA Nemotron Coalition, contributing post-training infrastructure and 2,500-plus reinforcement-learning environments, with Nous Research and Pluralis training by the same logic.

Will the Open Stack Reach the Frontier?

The open stack can probably reach last year’s frontier, and on post-training it is already there. Reaching the live frontier is harder, and the obstacles are pre-training scale, private data, and funding rather than post-training compute.

“The frontier” can mean two different things. If it means the best model from 18-24 months ago, decentralized training is close enough to take seriously. Money is now the binding constraint: can these projects keep paying for large runs once the coordination problem looks solved?

If it means the best available model at any given point in time, the answer is probably not this decade, and possibly not ever. Distributed training can only gather so much spare compute, and the leading labs are still accumulating more.

There’s also recursion to take into account: the frontier improves every year in part because the labs use their own models to help build the next ones, so the pace of improvement quickens.

The strongest case for distributed training is efficiency. The compute required for a given level of capability keeps falling, and DeepSeek trained a competitive model for roughly a twentieth of a comparable budget. If that continues, decentralized training won’t need to match the frontier’s raw compute directly, because the amount of compute needed to reach a useful level keeps falling.

The harder limit is everything that has not come loose. The base models still come from a few frontier pre-training clusters, and the frontier’s edge increasingly sits in proprietary interaction data and in reward design for fuzzy, hard-to-verify work, where you cannot simply check the answer. Open post-training bites hardest in verifiable domains like code and math, and least where the reward itself is the moat.

Funding is also unresolved. No legitimately durable way to pay for community-scale training has emerged, so even if the technical path works, the money for the largest runs may never materialize.

The Common Property

Open weights, local inference, and distributed training are at different levels of maturity, but they share the same basic property: they reduce dependence on permission from a frontier provider.

The frontier will always be important, but it may also become inaccessible, illegal to use, or nationalized outright. As long as they survive, the largest labs will preserve advantages in capital, infrastructure, data, talent, and post-training, and many users will keep paying for the best hosted models when the work justifies it.

But the hosted frontier no longer has to be the whole stack. A company, country, or individual can use frontier access where it’s most valuable while keeping enough capability outside the hosted channel to survive disruptions in service.

After Fable, that’s the standard a serious, sovereign AI stack has to meet: when a provider stops granting access, useful AI must live on.

I can’t say I’m surprised. On April 29th I described three tiers of intelligence forming: a frontier the labs keep for themselves, productized commercial models in the middle, and open-weights models at the bottom.

Governments, I wrote, would demand the frontier “primarily for military and national security reasons, which they’ll eventually be granted whether the labs do so willingly or not.”

More Than Moral

I have championed open technology for years. I have invested in it, built it, and written about it.

Open technology often gets invoked as morally superior to closed technology, but morality is not why it wins.

I’ve seen this firsthand: principles don’t drive adoption. Users won’t choose open technology if the product isn’t better, and slogans like “open technology is better for humanity” aren’t business strategies. Open technology cannot endure exclusively for moral reasons. It must also add value.

Thankfully, it adds value because it’s cheaper, more accessible, more controllable (”sovereign”), and more resilient under constraints. When constraints aren’t the priority, these qualities are usually ignored. But the moment constraints become apparent, “good enough” open alternatives come roaring back into focus.

I called this dynamic MACHA, Make AI Cheap Again: if the first wave of AI was about capability, the next is about efficiency. In a MACHA world, “good enough” compute is a feature, not a flaw. I made the case that increasing economic pressure, geopolitical tensions, and a never-ending battle for control (all different forms of constraint) would end AI’s “scale at all costs” era and force a turn to cost-per-token discipline.

Sam Altman has conceded that token costs went from a non-issue to "a huge issue" inside a single quarter, and enterprises now route the bulk of agent work to cheap open models, reserving frontier calls for the hard problems and cutting costs by as much as 95%. If the gap between closed and open is that wide, someone will inevitably turn it into a business model.

Free AI is ending and tokenomics is beginning: we're now squarely in a MACHA world.

Who Holds the Switch

The Mythos suspension exposed an asymmetry: a closed model can be revoked, but an open one can’t.

Washington could switch off Mythos and Fable because they were gated, running through a single provider. No one can switch off DeepSeek, Qwen, or what’s left of Llama. Once the weights are out, they sit on ten thousand drives that no government directive can reach.

Only open models whose weights are transparent and verifiable offer control over the corresponding intelligence. Anything short of that involves trusting the provider, and as Anthropic demonstrated: trust takes a lifetime to build and an instant to break.

Dean Ball warned that training a soon-to-be-smarter-than-you system to deceive its own user is different than a model that refuses out in the open. Christian Catalini looked further down the road, to the decentralized answer a single chokepoint tends to summon:

From Anthropic’s original choices to Washington’s reply, the entire episode was an argument about who decides.

Back in April, when Mythos was announced as too dangerous to release, AISLE found that small, cheap, open-weights models recovered most of its showcase vulnerabilities, including the flagship FreeBSD exploit, caught by a model with 3.6 billion active parameters at eleven cents a million tokens.

The moat, they concluded, is the system, not the model. The week Fable shipped, OpenRouter combined a panel of budget models to achieve performance within one percent of the frontier on a research benchmark at half the cost. You can, indeed, revoke access to a model, but its capability is already available, and a weekend of open tooling can put most of it back.

Superabundance or Apocalypse

It’s worth noting that the US Government’s action is an immediate consequence of the hyperbolic debate surrounding frontier artificial intelligence: according to its leaders, AI either leads to superabundance or apocalypse.

Neither scenario is actually plausible, but it’s commonly employed rhetoric because it’s easy to communicate, easy to understand, and it animates both the public (which is useful to politicians) and investors (which is critical for fundraising).

Anthropic has portrayed Mythos as a dangerous, weaponizable model since its inception, and in a matter of days the US government agreed to treat it as such.

I’ll grant that the danger is real. Simon Willison spent two days with Fable and called it relentlessly proactive: give it a goal and it invents tricks nobody has written down to reach it, treating every constraint you set as a problem to route around. That makes it enormously useful to someone with a hard problem to solve, and dangerous in the hands of someone who wants to cause harm.

If you believe a model can meaningfully uplift a bad actor, gated and revocable access starts to look prudent. But the prudence and the power grab are the same. The question was never whether the technology is dangerous. It was who gets to hold the off switch, and whether they will keep their word. Concentrating the switch doesn’t make the danger go away. It moves it from many hands to very few, and leaves one company, or one government, deciding what billions of people are allowed to compute. That is the larger hazard, and it is the one open weights answer.

The Unwilling Hero

A word on Dario Amodei. There is a scene in Gladiator where a dying Marcus Aurelius offers Maximus the chance to be emperor of Rome.

“There is one more duty I must ask of you before you go home,” he groans. “I want you to become the protector of Rome after I die…Do you accept this great honor I have offered you?”

“With all my heart, no,” replies Maximus.

“That is why it must be you,” Aurelius asserts.

This is the image of an unwilling hero, popularized as the refusal of the call in Joseph Campbell’s Hero’s Journey.

Dario Amodei has spent the last few years desperately trying to convince the public he is Maximus refusing the call. He would have you believe he built Anthropic reluctantly, to take responsibility for a dangerous technology everyone else would misuse.

For what it’s worth, I think he’s sincere. But sincerity is beside the point, because it doesn’t reconcile with what he’s built.

Over the past year Anthropic has assembled one of the most ambitious commercial machines in technology, from the $100 million Claude Partner Network and Project Glasswing in cybersecurity to an enterprise services joint venture with Blackstone and Goldman and the $150 million Claude Corps placing fellows inside nonprofits.

He’s built a universe aimed at embedding Claude into the economy, cybersecurity, and philanthropy.

As I argued in Models Aren’t Moats, value does not collect in the model. It collects in distribution and lock-in, which is what this machine is built to manufacture.

This is the part that smacks of CS Lewis’ “benevolent tyranny”:

“Of all tyrannies a tyranny sincerely exercised for the good of its victims may be the most oppressive…those who torment us for our own good will torment us without end for they do so with the approval of their own conscience.”

The problem is Anthropic is trying to "thread the needle," casting itself as a self-aware, safety-conscious frontier lab at the same time as it pursues one of the most ambitious for-profit commercial endeavors ever attempted. The market it competes in will, sooner or later, force the very compromise the mission was meant to prevent.

Long Live Open Technology

The case for open technology is age-old, time-honored, and the record is clear.

As we wrote last April, modern computing was built on open foundations, from UNIX and Linux to Python and Git, and today’s giants owe their scale to freely accessible, permissionless software.

Open technology is founded upon principles of liberty, freedom, and basic human expression. A technology controlled by someone else (an individual, a corporation, or a government) can be changed, monitored, priced, or withdrawn at their discretion, and the more it matters, the more that power is worth.

Open technology is no more or less than a simple and effective counterbalance to power: a natural response to forces that would upset an equilibrium.

This is not only the open-source faithful talking. Bruce Schneier, writing the same week, reaches an AI public option as the answer: fund open harnesses that balance capability against safety, and open models whose provenance is public.

AI is a general-purpose technology, and whoever controls it stands to gain enormously, which is why the fight to control it is on. Anthropic spent years trying to establish itself as the lab everyone could trust. In mishandling the public release of Fable, it forfeited part of that credibility and ushered in the era of “Government-controlled AI” in one fell swoop.

The response was fast and collective, the way an immune system meets a virus: the people who care about AI saw a power grab and moved against it.

Power gathers wherever it is allowed to. Open technology is what checks it, not because it is virtuous but because it cannot be taken back once it is out. That is why it endures, and why it will outlast whoever is holding the switch this week.

At 5:21 in the evening on Friday June 12 2026, the US government sent Anthropic a letter. Citing national security and export-control authority, it ordered the company to suspend all access to its two newest models, Fable 5 and Mythos 5, for every foreign national in the world, including Anthropic’s own foreign-national employees. To comply, Anthropic had to switch both models off for every customer, everywhere. By its own account, the company learned the directive existed and had to start pulling the models down inside the same evening.

Anthropic published a statement and posted it the same night. The operative passage:

We are complying with the government’s legal directive and are removing access to Fable 5 and Mythos 5 for all users. However, we disagree that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people. If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.

As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles.

The stated reason was a jailbreak. The government had seen a technique for getting past Fable’s safeguards, and according to Anthropic the technique amounted to asking the model to read a codebase and point out its flaws, a capability the company says is already available in other deployed models including OpenAI’s GPT-5.5 and used every day by the people who defend software for a living. Anthropic is complying, and disputing, at the same time. It says recalling a model deployed to hundreds of millions of people over a narrow finding is disproportionate, and that if the standard were applied evenly it would halt new model releases across the entire industry.

They are right about that. I want to be precise that they are right, because what follows is not a complaint about Anthropic. It is a description of a machine, captured in the moment it was switched on.

I have been making this argument since at least April but I’m honestly surprised it came this fast and in this manner. In Artificial Good Enough Intelligence (April 28) I argued the frontier was going closed, that the best models would increasingly be withheld, restricted, or selectively exposed. In AI Waves #6 (April 23) I wrote that the closed labs had begun to gate capability by trust level, not just price. What I had wrong was the hand on the gate. I took it to be the labs, withholding the top tier as a business decision. Friday the state reached in and took the gate for itself. The off switch was always the risk. It just changed hands.

The Reversal Test

On Wednesday, in Policy on the AI Exponential, Dario Amodei proposed that frontier models pass mandatory third-party testing and that the government hold the power to block or reverse a release that fails. He asked for this explicitly, as the responsible path, with the caveat that the power be exercised through a process that is transparent, fair, clear, and grounded in technical facts.

On Friday, a release was reversed. Against him. Through a process that, by his own account of it, was none of those four things. The letter arrived without specific technical detail, gave one evening to comply, and rested on a finding the company considers trivial and widely available elsewhere.

The gap between Wednesday and Friday is forty-eight hours, and it contains the whole argument. The power to reverse a deployment is not safe in proportion to how carefully it is designed. It is dangerous in proportion to the fact that it exists. Anthropic asked for an instrument and was handed the sharp end of it two days later, which is the oldest lesson in the book about building machinery and assuming you will be the one holding it.

You Cannot Recall Open Weights

Here is the part that matters past this weekend. The reason a directive could darken Fable and Mythos in an evening is that they are closed. They run where the company can reach them, which means they run where the government can reach the company. A single letter to a single legal department took two models away from hundreds of millions of users at once.

There is no equivalent letter for open weights. Once a model’s weights are public, they sit on tens of thousands of machines in dozens of jurisdictions, and there is no address to send the order to. You can regulate what people do with them, you can prosecute misuse, but you cannot recall them. The off switch does not exist, because there is no single hand for it to sit in.

This is the entire case for open models, and it has nothing to do with whether open or closed is safer in the ordinary sense, the sense of refusing a dangerous request. It is a point about who holds the switch. A closed model concentrates capability in a way that is efficient, governable, and revocable. Friday was a demonstration that revocable means revocable by someone other than you, for reasons you may consider trivial, on a timeline you do not control. Open weights are messier, less governable, and harder to make safe in the ordinary sense. They are also the only architecture with no switch to seize. You trade one kind of safety for another, and Friday clarified the price of each.

The Counterweight

This is why the open layer is not an ideological preference. It is a structural one. A world with only closed frontier models is a world with a small number of off switches, held by a small number of parties, that turn out to be live. The counterweight to that is not a better-behaved lab or a wiser regulator, both of which require the institution to stay good forever. The counterweight is infrastructure that does not have an off switch in the first place: weights anyone can run, compute anyone can rent, verification anyone can check. Capability that no letter can recall.

I spent two essays arguing that the threat worth pricing was the concentration of control, not the awakening of the machine. I did not expect the argument to be settled this fast, or this literally, or at Anthropic’s expense rather than in its favor. Friday was not a story about a dangerous model. Every account, including the government’s, agrees the model was not the danger. It was a story about a switch, and the discovery that it works.

p.s. I’m printing these t-shirts. If you managed to trip the safety limits in the last 72 hours you get one. Reply here and I’ll send you one or feel free to copy the idea.

This past Friday President Trump announced he was considering having the government take equity stakes in the largest AI companies, with the proceeds routed back to the public as dividends. No stranger to controversy, breaking tradition, or pursuing strategies historically considered anathema, the President has already directed the government to purchase about ten percent of Intel, ostensibly on the public’s behalf.

Never one to sit out a discussion about wealth redistribution, even Bernie Sanders has proposed a bill for fifty percent federal ownership of the AI megalabs in question. It is not every day that Trump and Sanders reach for the same lever.

In the hallowed words of Oscar Wilde, life imitates art far more than art imitates life. These forecasts have become the genre through which we read the present, and we’ve stopped asking which kind of film we’re in.

AI 2027 and Situational Awareness

Nationalizing frontier AI labs was predicted more than a year ago by a scenario titled AI 2027, a detailed, month-by-month narrative forecast of how AI could progress from today’s agents to superhuman AI and superintelligence by around 2027. Leopold Aschenbrenner’s Situational Awareness, published in 2024, also predicted nationalization, claiming that AI would eventually become a matter of national security.

Both sets of authors have been remarkably successful in their predictions thus far, and they deserve credit for the quality of their research. The AI 2027 authors published their own self-evaluation, and other independent websites now exist that attempt to track its predictions one by one. The capability and safety predictions are the ones tracking closest, several of them early, which is exactly the point: the question is not whether these systems keep getting more capable, it is what that capability turns into.

Aschenbrenner was recently profiled in the Wall Street Journal and boasts impressive returns on investments made on the convictions in his manifesto.

But how do we go from powerful technology to nationalization in a country so wedded to free-market capitalism? How is it that both Republicans and Democrats publicly agree it is the right course of action in the midst of such an aggressively polarized political environment?

Superabundance or Apocalypse?

Since the launch of ChatGPT there has been a steady drumbeat of headlines, proclamations, essays, books, and podcasts all pointing to only one of two hyperbolic outcomes for AI: superabundance or apocalypse.

If the headlines are to be believed, we are either headed for paradise or AI kills us all.

Presenting technological revolutions as the difference between life and death is common. Everyone is interested in the future, and clear, simple outcomes are easy to communicate and easy to understand.

Simple, extraordinary, hyperbolic outcomes are also evidently a fantastic fundraising strategy, because it is hard to raise a hundred billion dollars for a merely useful tool.

Even before founding Anthropic, while still at OpenAI, Dario Amodei was wary of releasing GPT-2, judging it too powerful to put out. He has since written the paradise version in Machines of Loving Grace and the abyss version nearly everywhere else. Demis Hassabis, the field’s designated good guy, keeps sounding the alarm too.

The Messy Middle

But clear and simple outcomes rarely materialize, and grappling with the more probable messy middle means entertaining complexity, nuance, and uncertainty. That takes unusual technical literacy and several rare personality traits most people do not have. Much of this technology is genuinely unprecedented, the frontier is moving, development is accelerating, and diffusion is uneven. So very few people have the tools to evaluate a hyperbolic headline, which is exactly what makes the headlines work.

I made this point last October. We are confidently building toward a future we cannot picture. Eric Schmidt and his peers forecast crisply to 2030, maybe 2035, then simply stop, because no one has a framework for the other side. The honest position, the one that does not trend, is that nobody knows.

But nobody knows is not the same as no forecast. The honest forecast is just a less dramatic one, closer to a couple of points of GDP than to heaven or extinction. Declining to pick paradise or apocalypse is not declining to predict. It is declining to predict theatrically.

The set of plausible futures has exploded, the frontier feels almost infinite, and grappling with infinity is how valuations detach from reason and people start invoking God when they mean machine intelligence.

Suspending Disbelief

The primary problem with these frameworks is the massive, albeit subtle, logical leap they require to go from powerful technology to existential threat to humanity.

The leap is a hallmark of science fiction, and this is the scene where the film asks for it. To lose yourself in Battlestar Galactica, Star Trek, Star Wars, Dune, or Ender’s Game, you first agree to stop asking whether the world on screen could exist. The forecasts ask the same of us, and most of us oblige.

To be clear, the fear of runaway adversarial intelligence is rational. We are right to be afraid of runaway rogue intelligence. But it is misdirected, because it rests on an unfounded assumption. Hidden behind all of the rigorous terminology involving FLOPS of compute, orders of magnitude, exponential development curves, and real, observable trends, there is a simple leap: increasingly capable AI begets self-directed AI.

We’re being invited to believe that really, really good software leads to self-directed, autonomous machines that will want to do good things or bad things depending on how well they are aligned with humanity.

At face value the leap is not immediately evident, which is why it is easy to make. Indeed, long-running agents are already effectively self-directed. But they are self-directed in the loose sense we described in Much Ado About Autonomy:

It is true in a loose sense that agents are programs that can act independently. As I wrote a few weeks ago in AI Waves, an agent is a tool that wields its own tools, does its own work, and has the capacity to operate on its own.

Loose autonomy is independence in execution: humans identify objectives and delegate agents to execute against them. It is useful, valuable, and applicable today.

But autonomous agents also implies a stricter, maximalist form of autonomy: an agent originating its own objectives and operating on its own authority.

The strongest version of the other argument does not need the leap at all. It points at a measured trend: the length of a task an agent can finish on its own is doubling every few months, and Anthropic reports its own models now write most of its code. Extend the line, the argument goes, and the human supervisor quietly disappears. But a longer leash is not a different animal. A model that runs unattended for twelve hours is doing delegated work for twelve hours, not choosing its own ends. What is growing is loose autonomy, execution stretched over a longer horizon. Strict autonomy, originating the objective, is a separate claim, and a rising horizon is not evidence for it.

It’s possible that agents develop strict autonomy, but it is not probable, and it is certainly not inevitable. The line between the two is not clean. A delegated goal spawns its own sub-goals, and a system told to improve itself is, in a narrow sense, choosing what the next system optimizes for. Whether that gradient produces an adversary or simply a more capable instrument is the real question, and it deserves its own essay. The leap is not proven either way. It is just routinely assumed in one direction.

AI is getting extremely good, better than humans, at programming, but despite all of the colossal investment and trillion-dollar valuations, good programming does not necessarily generalize to powerfully adversarial self-directed agents. Great programming will have massive implications for many things including science, medicine, math, and software, all of which will greatly impact society, but it does not have to mean that machines become self-directing, and it does not have to follow that AI is an existential threat.

Asymmetric Information

Artificial intelligence is distinct from other transformative technologies in one respect: it empowers both individuals and nation states.

What AI spokespeople claim when they present the situation is that incredible technology may one day become more powerful than humans. That was true, for example, of nuclear weapons.

Individuals cannot wield nuclear weapons, however, whereas they can use AI to help develop bioweapons, which is why OpenAI and Anthropic, alongside a roster of scientists, signed a letter this month urging lawmakers to tighten screening of the synthetic DNA sequences that could be used to build them.

Asymmetric warfare is a good analogue too. Whereas much of nation-state military spending is focused on increasingly powerful and expensive weapons development, low-cost, unmanned drones powered by advanced AI and deployed by small groups have completely and irreversibly redefined modern warfare, as demonstrated in Ukraine and the Middle East.

The Individual Matters Most

Which brings us back to the government. Perhaps governments are more concerned with nationalizing AI so that they are equipped to protect against all enemies, individual or collective. Concentrated, already-dangerous capability exists, and governments have decided it is already strategically decisive.

Investing in Intel was ordinary industrial policy, a strategy to own a piece of the infrastructure buildout. Reaching for the model companies themselves, on the other hand, is more direct-to-consumer: whether the models become autonomous, conscious, or rogue does not matter.

We’ve been led to believe that the primary threat involved in AI is rogue intelligence, Terminator-style. But it is much more likely to be rogue humans who learn to wield AI in novel, dangerous, and unpredictable ways.

The difference is not cosmetic. If the danger is a rogue machine, the response is to control the model: align it, cap it, build the kill switch, perhaps own the weights outright. If the danger is a rogue human holding the machine, the response is to control access and the chokepoints: who can point the tool, at what, and through which supplier of compute or synthetic DNA. Misdirected control is not a smaller problem than the one we are bracing for. It is the same problem aimed at the wrong target.

By the same token, AI-enhanced operators with good intentions are now equipped to do things that were not possible before. As an investor and an optimist, I am focused on this last reality, but it is just as important not to underwrite paradise as a business plan. Many AI-skeptics believe in the technology and doubt the valuation. It is possible that OpenAI, Anthropic, and the rest build genuinely transformational technology without becoming enduring businesses. The dot-com build-out was real, most of the companies that ran it did not survive it, and the fiber they laid is still in the ground. Transformational and enduring are different words.

If we eschew the hyperbole, AI is a transformative technology. Period. Through that prism you can see where value actually accrues, and the dark fiber is the warning. The rails get overbuilt, so owning raw capacity is not the same as capturing value. What lasts is the scarce layer: the work of improving the models, the verification that makes heterogeneous capacity trustworthy, and the operators who wield it all best. The boom always lays down too much supply and too little of that.

We are surrounded by narratives that jump from capability to civilizational destiny in a single bound. Life imitates art, but we have been bracing for the wrong genre. The blockbuster ending is the part that needs the suspension of disbelief. The story actually unfolding is the docudrama: the people this technology makes powerful, the people it makes dangerous, and the fight, already starting, over which is which.

As Karpathy wrote, agents actually crossed the chasm in ~November/December of 2025.

What’s more? As Ethan Mollick notes, the AI labs have generally been right. It is extremely tempting to judge the labs and their predictions almost instantly (see Dario’s prediction that AI will render knowledge work obsolete in the near future…). It doesn’t help that enormous amounts of money depend on these judgments and their accuracy, either… but maybe we should *try* thinking a bit longer-term.

OpenClaw

In any case, by all accounts, 2026 will, in fact, be the year of the agents, and Exhibit A is OpenClaw.

Much has already been written about OpenClaw. I will not be contributing another explainer. For a comprehensive overview, please see here.

If you aren’t already aware, OpenClaw is an open-source, autonomous agent created by Peter Steinberger that runs locally on your machine, connects to your messaging platforms – WhatsApp, Slack, Telegram, Discord, iMessage, Signal, Teams, etc – and operates as a persistent daemon: always on, always listening, always acting.

Apparently, Steinberger was personally losing $10,000 to $20,000 per month to keep it running. In any case, it’s no exaggeration to say that it has ROCKED the AI world, in every possible way. Reactions range from measured enthusiasm because agents finally demonstrated utility to the typical breathless excitement over technology deemed “life changing.”

Although creating it is an immense achievement worth recognizing, the core technology isn’t actually revolutionary. It’s a well-designed wrapper connecting an LLM to local tools and services. What made it take off is that it put familiar pieces together in a way that felt, for the first time, like having a genuine digital assistant rather than a chatbot you open and close. It’s what everyone was hoping for in 2025.

As such, traction and usage exploded. To speak of product-market-fit is a dramatic understatement: as with nearly every popular AI-native invention, adoption curves don’t resemble anything else we’ve ever seen in history. OpenClaw crossed 100,000 GitHub stars faster than React, TensorFlow, or Kubernetes. The adoption curve is literally vertical.

There is perhaps no better demonstration of its importance in the AI landscape than the end to the beginning of this fascinating development: joining OpenAI. Yupp! Sam hired him within 3 months.

The Problem

As you can imagine, with great power comes great responsibility, and most people don’t care about the responsibility part.

Part of the reason OpenClaw is so effective is that it operates with the full privileges of its host user. That means shell access, file system read/write, OAuth credentials for connected services, and any digital access keys a user is willing to give it. For that reason, users began buying out dedicated Mac Mini computers for a “clean” environment on which to run their instance of this software.

Meredith Whittaker, President of the Signal Foundation, had already articulated the fundamental tension involved with giving agents root access to computers at SXSW last year.

Nevertheless, tens of thousands of people gave an autonomous AI agent root access to their computers.

Source: https://www.wired.com/story/malevolent-ai-agent-openclaw-clawdbot/

VentureBeat put it best: “OpenClaw proves agentic AI works. It also proves your security model doesn’t.” As a quick overview of the security hazards involved 😂 :

• Security researchers found 40,000 exposed OpenClaw instances on the public internet

• BitSight deployed honeypots and saw exploitation attempts within minutes.

• Because SOUL.md – the file defining the agent’s identity – is writable, prompt injection attacks can achieve persistence: an attacker tricks the agent into rewriting its own instructions, creating a backdoor that survives restarts.

• ~7% of the skills available on Clawdhub were found to be malicious – installing stealer malware that harvests browser passwords, crypto wallets, and session cookies

• And much, much more…

CrowdStrike released an enterprise-wide security framework for OpenClaw. Cisco’s security team called it “groundbreaking” from a capability perspective and “an absolute nightmare“ from a security perspective.

The Opportunity

All of which is to say there’s an opportunity here. For what it’s worth, I’ve seen this before. At Sun Microsystems, proprietary infrastructure migrated to commodity hardware and Linux. When that happened, companies like Red Hat built extremely valuable businesses implementing frontier software for enterprises.

In fact, enterprises will pay enormous sums for someone to make free software reliable, secure, and compliant.

Chamath mentioned as much on the All-In podcast a week ago, saying that “on-prem is back:”

“Once you use these tools, it is very difficult for a company to be able to control how their data is used subsequently thereafter… If you’re using a set of agents to act on all that information, all those agent traces are going back to these model builders... When they realize that that is a problem, the enterprise will have to decide, do I just give up and keep running all of this stuff in the cloud in a shared experience or do I bear the incremental cost of running this stuff in a more coordinated manner that I control on prem?”

Supporting his claim, Chamath pointed to Judge Jed Rakoff of the Southern District of New York’s ruling in United States v. Heppner that said “documents generated through a consumer version of Anthropic’s Claude AI were not protected by the attorney-client privilege or the work-product doctrine under the circumstances presented.”

There is also data suggesting that 86% of CIOs now plan to move some workloads back on-premise, and Lenovo’s 2026 TCO analysis found on-premise AI inference achieves breakeven in under four months versus cloud.

So is there an AI-native version of Red Hat? (Reach out if you’re building this!) We think it may involve something like Palantir’s “Forward Deployed Engineer” model. For a brilliant breakdown of the FDE, see Marty Cagan’s piece here. There are a few highlights:

What makes Palantir’s model work: FDEs build prototypes on top of Palantir’s platform. A platform product team then generalizes what FDEs learn across clients into reusable capabilities. This prevents bespoke sprawl while keeping the deep customer intimacy.

The broader principle: send your best engineers to sit with multiple customers, see the problem space firsthand, and discover solutions that serve real needs across them. This is customer discovery at its most effective.

It goes without saying that there’s lots of data demonstrating that this model works, too (see $PLTR’s market cap, their margins, and their growth). Palantir’s FDEs embed directly with clients to configure and deploy their own software against real operational problems, but it’s no stretch to imagine some cracked team of devs organizing to form a modern form of elite AI-consulting configuring bespoke on-prem instances of the latest AI. In fact, Cagan breaks down how and why traditional consulting models (specifically Accenture) break down when faced with AI.

As such, the AI-native, Red-Hat-meets-Palantir model could become especially valuable because understanding the customer environment in the context of AI is more complex and critical than ever.

To be clear, that model is as follows:

1. Red Hat for the software layer

   1. Open-source, or commodity agent frameworks as the core, enterprise subscriptions for security, compliance, and support

2. Palantir (the FDE framework) for deployment

   1. Embedded engineers who configure agents against specific data, workflows, and regulatory constraints

Cloud isn’t dying, either. What’s happening is selective repatriation: enterprises are moving their most sensitive, high-utilization AI workloads on-premise while keeping burst and experimental workloads in the cloud. But for many organizations (including nations!), control is all that matters. As autonomous agents handle more and more proprietary data, the trend is clear.

Nazaré’s Position

The industry’s consensus remains scale – more compute, larger models, bigger data centers, etc. But other problems exist, the investment space is less crowded, and returns are therefore (much) more attractive. The “who, what, where, why, and how” for AI is critically important, especially within real organizations.

OpenClaw means that 2026 will be the year of agents. We know that models like those of Red Hat and Palantir for enterprise open-source and embedded deployment are feasible and lucrative.

We’ve believed for some time now that, among many other opportunities in AI, the company that configures, deploys, secures, and maintains autonomous AI agents on sovereign infrastructure will build one of the most important businesses of the next cycle.

If you’re working on that, we’d love to hear from you.

7@nazare.io

nazare.io

Welcome to another edition of Robot Wave, where I explore trends in AI.

This has been a big week for AI. Not only is 2025 shaping up to be the year of the agents, but it may also be the year that we Make AI Cheap Again (MACHA)?

Late last year a Chinese company Deepseek released a new model which claimed to use significantly less compute to train than similar models from OpenAI and Anthropic. The latest release on 20th Jan 2025 claims to have similar results to ChatGPT o1 models, again with significantly smaller compute for training. So what’s going on?

Let’s first rewind a bit. I first introduced the concept of MACHA a year ago. What struck me at the time was how the entire AI industry was moving in one very similar direction - just add more compute, more data and of course more $ to the same architecture. The concept of innovation on the model (an area I was trained in through my research in the 90s) was mostly ignored in favor of more compute and more data (as an side note, my PhD was focused on scaling neural network training using Mixtures of Experts). The concept of “scaling law”, where to get better performace, more data and more weights / compute are added to a system has resulted in fantastic performance such as OpenAI and Anthropic but also fantastical investments ($1bn seed round for Ilya’s SSI company, XAI 100,000 H100 cluster etc).

“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation. There were many examples of AI researchers' belated learning of this bitter lesson, and it is instructive to review some of the most prominent.”

Rich Sutton

The entire enterprise seems akin to a modern data AI Tower of Babel - a literal race to the top to meet God (AGI).

““Come, let us build ourselves a city and a tower with its top in the heavens, and let us make a name for ourselves; otherwise we shall be scattered abroad upon the face of the whole earth." The Lord came down to see the city and the tower, which mortals had built. And the Lord said, "Look, they are one people, and they have all one language, and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them.”

— Genesis 11:1–9

Commodity hardware and software eats the world

Back in the year 2000, after finishing 3 years of consulting work at NASA Ames and exploring dotcom ideas, I joined a small company called Infrasearch which was started by some of the early developers of Gnutella, an open source file sharing tool. Infrasearch developed search technology based on the ideas of Gnutella and new XMLRPC technologies. We tried to raised venture funding and even though our seed investors included Marc Andreessen, we ultimately were acquired by Sun Microsystems in 2001.

At Sun I initially ran the P2P research team, JXTA and then started a new project focused on archival storage using clusters of commodity AMD servers and commodity hard drives connected using self repairing and managing software.

What’s the connection to today you may ask? Well at the time Sun made the majority of its money from selling large expensive servers running closed source operating system software. Sun made most of its money from a small number of customers and had been extremely successful in the dotcom boom along with Oracle and EMC. During the dotcom boom I had been working on open source web and operating systems such as linux, mysql and php running on commodity servers. These approaches were often eschewed by investors and the startup MBA tourists who ran many of the dotcoms. In the end the scrappy commodity hardware and software designs were the winners. The impetus for the change, however, was a financial shock in the form of the dotcom bubble bursting. This moment led IT managers to question the cost of the Sun and other machines and instead try machines from Dell or other players using Linux and the open source web stacks. A similar event occurred after the 2008 GFC leading to a move to cloud computing away from self hosted infrastructure.

Deepseek

I believe a similar event is happening now. Deepseek has blown open the door to a Pandora’s box of open source AI technologies which was first opened by the “leak” by Meta of Llama, the open source LLM in 2023.

Our investment in Vast tracked this change last year. Vast provides mostly consumer grade Nvidia chips in a GPU rental market. Vast is growing rapidly and while most applications are focused on inference, new technologies such as Deepseek and also distributed training systems such as Nous Research, PrimeIntellect (Nazare portco) and Gensyn may end up disrupting the current training paradigm. This well written piece presents the hardware and software case against Nvidia (more), whilst Musk and others are questioning whether Deepseek was really trained using so few GPUs. The code is open source and some researchers are already reproducing the results.

Deepseek combines 2 things we are already familiar with (see this review also)

• Mixtures of Experts (MOE) architectures in which sections of the billions of weights are turned on and off during training and inference for efficiency,

• Quantization: using KV cache compression to make the training and inference more efficient.

The final innovation is to combine these first 2 optimizations with the Chain Of Thought reasoning of OpenAI o1 and a Reinforcement Learning approach. This final optimization seems to be the technique that results in the reported dramatic ($5.6m) reduction in cost of training (compare this to the 100k H100s of XAI).

Naturally people are skeptical, and many researchers are attempting to reproduce the results. Some people are suggesting its a Chinese psyop and China has possession of H100s, thus violating the US sanctions.

I am in the camp that this is a real innovation not a fake one. However it plays out, the genie is out of the bottle that the perception is that we can do more with less - we can train and execute large models using less hardware, potentially commodity hardware and without the incredible high cost of investment in hardware infrastructure.

What’s next?

Do the massive clusters of Nvidia’s most powerful chips now become irrelevant? Not necessarily. Even as the LLM market changes dramatically with less reliance on compute, the quests for ASI and AGI – humanity’s Babylonian pursuit of finding “God” – will likely still be dominated by throwing maximal resources at the challenge. Over time, however, I expect that software improvements will keep strong moats from developing in these areas just as we’re beginning to see happen with LLMs. This is now a brave new world. What was taken for granted is being questioned, prior assumptions are being challenged, and new possibilities abound.

Till next time,

Steven

Dr. Steven Waterhouse

Founder and General Partner,

Nazare Ventures

X, Linkedin