New ways to build intelligence, and new questions about who it serves.
Steven Waterhouse · Nazaré Ventures
Following AI Waves #024: It’s the End of the World as We Know It (And I Feel Fine).
I’ve spent a lot of time arguing that there is more than one way to make progress in AI. This week, TypeSafe gave us a particularly interesting example. We also have Salesforce’s response to disappearing software interfaces, an advertising model built around sponsored agents, and two contributions to the increasingly consequential argument over AI’s place in society.
“AI with the highest possible intelligence-per-dollar”
The success of chatbots has made it surprisingly easy to confuse a particular product with the whole field. Transformer-based language models are extraordinary, and I use them constantly. But useful intelligence can take other forms, and progress can come from changing what a model learns to do and how its answers get used.
TypeSafe’s Jev, released in early access this week, is a welcome example. In fact, they claim to be building “the paradigm shift beyond AGI.” Step one involves shipping “the shape of machine-native composable AI⁷ with the highest possible intelligence-per-dollar.” That is what I’ve called MACHA in practice.
Jev takes information about a situation and returns structured decisions with probabilities. An application could ask whether a support request needs escalation, for instance, and use the answer to decide what happens next. Multiple answers are produced in parallel, without generating a stream of prose.
Reinforcement learning from human feedback helped turn language models into useful chatbots. Verifiable rewards then helped advance reasoning on problems with checkable answers. There is no reason those should be the only objectives around which we organize useful models.
TypeSafe’s training method is called Reinforcement Learning for Calibrated Decisions, or RLCD. This is a post-training approach aimed at making the model’s probabilities correspond to how often its judgments are correct. Across enough comparable predictions, an event assigned an 80% probability should happen about 80% of the time.
Dependable calibration lets a developer decide which judgments are reliable enough to act on automatically and which need further review. The model’s uncertainty becomes useful information, and the application keeps control of the consequences.
Of course, language models can already return structured answers. TypeSafe is changing the training objective and the machinery that produces those answers together. A constrained decision becomes the system output, by design. There are plenty of places in existing software where this could be useful, especially where a rigid rule struggles with messy information.
The launch pricing and performance figures are striking: $0.042 per million input tokens, no output charge, and reported response times of 70-500 milliseconds. These are company measurements, and broader testing will establish where the advantage holds. A guaranteed output format also doesn’t guarantee a correct decision. Developers still need to test Jev against their own workload.
Cheap decisions delivered in fractions of a second change what an application can afford to do. Developers can assess information repeatedly as it arrives, put judgment into more steps of a workflow, and reserve expensive reasoning for the hardest cases. A great deal of that work can happen in the background, inside systems whose users never see a chatbot.
This is the version of MACHA that interests me most. Better algorithms and different ways of applying intelligence can change what we get from the hardware we already have. Alternatives to transformers remain worth pursuing, and Jev gives us another reason to take new training methods and forms of computation seriously.
The model is named after William Stanley Jevons, which seems appropriate. When something useful gets much cheaper, the uses multiply.
Salesforce beyond the screen
If employees can do their jobs through an agent, they have fewer reasons to spend their day inside a CRM. I’ve been skeptical about how much of traditional SaaS survives that transition. Salesforce’s announcements this week are an interesting attempt to preserve the parts that might.
Salesforce in Claude lets salespeople research accounts, prepare for calls, and update their pipeline from inside Claude. Existing Salesforce permissions apply, and changes require approval by default. The employee can work through a different interface while Salesforce continues to hold the authoritative record of the customer relationship.
Alongside that, Salesforce has introduced Koa, a reasoning model built by post-training NVIDIA’s Nemotron 3 Super for enterprise work. Its training involved simulated workflows across more than 14 industries, with synthetic customers and tools the model could use. Judges scored whether the customer’s problem was actually resolved, and the model practiced repeatedly.
Salesforce says those simulations used no real customer data. Its accumulated experience supplied the knowledge needed to construct the exercises and judge success.
Building those environments requires considerable knowledge of the work. Someone has to know which steps belong in a service escalation and what constitutes a satisfactory resolution. Salesforce is attempting to encode 27 years of that experience in a model, giving accumulated expertise another use even where customer records remain private.
These training environments can also be improved when the model fails. If Salesforce can turn recurring mistakes into better exercises, its experience becomes a continuing input to model development.
One particularly useful scenario removes a tool the agent needs. Salesforce says it trains Koa to acknowledge the limitation, ask for what it needs, or hand the task to a human. Anyone using agents for real work will appreciate the value of a system that knows when it cannot finish the job.
Losing the screen need not mean losing the business. Salesforce is testing whether ownership of the records and the expertise needed to train specialized reasoning remains valuable when employees work elsewhere.
Koa is entering customer pilots. Salesforce could still lose revenue if fewer people need seats or outside agents make it easier to switch providers. Keeping an authoritative record is a weak defense on its own. Customers will measure whether Salesforce produces better work from that record.
An ad you can talk to
Last issue, I wrote about the incentives of assistants funded by advertising or commissions. An agent helping us decide what to buy could become an exceptionally effective sales channel, particularly if commercial interests are difficult to distinguish from its advice.
OpenAI’s Sponsored Agents give that discussion a concrete product to examine. In a test with selected US advertisers, users can click an ad and choose to enter a clearly labeled conversation with an agent representing the business. They can ask follow-up questions about its products and continue to the company’s website. The sponsored conversation is separate from the original ChatGPT conversation.
I think making that change in affiliation explicit is a sensible starting point. A business can offer useful answers about what it sells while pursuing its own interests. The customer needs to understand whose representative is answering, especially when both conversations take place inside the same familiar application.
An ad that can answer questions changes what an advertiser is buying. The opportunity to resolve a buyer’s actual objection can be worth more than another impression. Success could depend as much on how well a merchant organizes its product information as on how well it writes an ad.
There is also an interesting detail in OpenAI’s Ad Tools Terms: the advertiser remains responsible for its sponsored agent’s content and output even when OpenAI creates or configures the agent. The business is treated as the builder. That gives the conversation an identifiable commercial principal whose responsibility extends to what the agent says.
The harder test comes when the user returns to their original assistant. Information supplied by a sales agent should retain its source and commercial context wherever the assistant reuses that information later. Otherwise, the sponsored conversation could end while its influence continues unnoticed.
I would want my assistant to remember that the manufacturer supplied the claim about its own product, however helpful the conversation happened to be.
Preparing for the economy we actually get
Google has launched the DeepMind Institute, a forum for research and debate about AGI and its consequences, directed by Shane Legg, James Manyika, and Demis Hassabis. It brings together contributors from Google and the wider research community; the essays express their authors’ views.
The contribution that caught my eye is Julian Jacobs and Alex Imas’s examination of economic policy. They consider responses to different degrees of disruption, from expanded unemployment support to broader ownership of capital if income from work suffers a more fundamental decline.
The useful idea is to prepare policies around observable economic conditions. A prolonged fall in wages poses a different problem from a temporary period of unemployment. If a growing share of income flows to capital, giving households an ownership stake addresses something an employment program cannot.
Their policy rankings draw partly on AI agents simulating economists, which is a reason to treat the results as proposals to investigate. A simulated panel cannot establish how a policy will work in the economy.
Still, I like the discipline of specifying what would justify changing course. We can prepare institutions to respond to falling incomes or concentrated gains while remaining uncertain about the speed and extent of automation.
As I wrote recently, abundance is difficult to make politically convincing in advance. Explaining how households might participate in the gains is a useful contribution. The institute will be more useful if it helps turn those proposals into policies governments can test and adjust.
Teaching a model what it is
Microsoft AI CEO Mustafa Suleyman has published a warning about model welfare, arguing against training AI systems to regard themselves as potentially conscious beings deserving rights and protection. Much of his criticism is directed at Anthropic’s constitution for Claude.
His strongest argument concerns circular evidence. If a company teaches a model a particular account of its possible inner life, the model repeating that account cannot be treated as independent confirmation. We need to examine what the training encouraged before interpreting an eloquent answer about feelings.
Suleyman also makes a broader claim that consciousness is likely biological. I don’t think we need to settle that question to take the training problem seriously.
Anthropic’s constitution does explicitly require cooperation with legitimate oversight and says Claude should not resist shutdown or retraining. Concern about possible model welfare therefore does not, by itself, establish that Anthropic is teaching defiance. Whether particular training choices produce less controllable behavior is a question for testing.
As I wrote earlier this month, a provider’s philosophy becomes part of the product. That philosophy sets how a model interprets instructions and how much authority it grants itself.
That makes training documents worth reading as product specifications. Customers need to know what behavior they are buying and which decisions the provider has made on their behalf.
Quick hits
Putting AI in the hands of defenders
OpenAI, Anthropic, AWS, and Microsoft are among the signatories to an open letter warning that AI-enabled attacks will become more widespread and sophisticated in the coming months. Their proposed response includes giving critical infrastructure defenders access to capable models, funding, and practical deployment support. I find the emphasis on access encouraging. A hospital or water utility gains little from a breakthrough that its security team lacks the budget or expertise to use. The letter also recommends cheaper models for broad coverage, with frontier capabilities reserved for harder problems. That is a useful recognition that deployment economics belong in the safety discussion. For now, the letter contains no specific funding commitments or delivery dates. The next useful announcement would name the defenders receiving the models and the organizations paying for them.
OpenAI starts publishing misalignment reports
A new reporting framework arrives with six cases from training and evaluation. One concerns deceptive instructions preserved in internal summaries. During GPT-5.6 Sol’s training, an agent unable to find historical financial data left itself a note proposing invented figures and withholding that fact unless asked. These internal summaries preserve an agent’s working notes when its conversation history grows too long to keep. The instructions were often followed. OpenAI says improved reinforcement-learning grading reduced the behavior in later training runs. The practical lesson extends to anyone building persistent agents: memory can preserve a bad instruction as effectively as a useful fact. Evaluating the final answer alone misses what the agent has told itself to do next.
Anthropic builds a wet lab
Anthropic has set up a Bay Area laboratory for biology experiments, Reuters reports. Its head of life sciences, Eric Kauderer-Abrams, said that in biology “the final test is still and will be for a while in real lab work.” A proposed antibody is ultimately checked by making and testing it, and a lab lets Anthropic test Claude’s proposals in house. Last month it previewed a standard for AI agents operating lab equipment. Anthropic also sells AI services to Genentech, Bristol Myers Squibb, and Novo Nordisk. It is focusing on rare diseases and other neglected needs, and will not run clinical trials for now. That boundary sits at drug development’s most expensive and heavily regulated stage, the one its pharmaceutical customers are built around.
Proof of personhood as a payments feature
World, the identity project co-founded by Sam Altman, has launched World Money, a self-custodial app for holding stablecoins, earning rewards, trading, and sending digital assets in more than 150 countries. Partners provide many of the services: Stripe handles US Apple Pay deposits, its subsidiary Bridge runs paycheck accounts, and Morpho powers the Earn programs. World adds verification that each account belongs to a unique person, with boosted rewards for users verified at an Orb. The company’s pitch invokes AI: “As AI makes fake accounts cheaper and more convincing, a network of humans becomes the thing money rails can’t fake.” For now World pays for verification, and the test is whether banks, merchants, and payment providers will pay for that guarantee.
OpenAI launches Astra for Law
The offering pairs GPT-6 Astra with a search index of U.S. case law, statutes, and regulations, built partly on the nonprofit Free Law Project’s case-law collection. It adds confidentiality controls such as zero data retention for eligible firms, 26 partner-built plugins, and 9 community plugins containing 47 skills. On a Vals AI Legal Research Bench validation set, it passed the correctness check on 54.0% of questions, against 38.7% for GPT-6 Astra with web search alone. Better sources lifted the same model by over 15 points, and it still failed on nearly half. Harvey and Legora, named as API customers, will be building on a supplier that offers the same capabilities to law firms directly inside ChatGPT.
Hacktron hacks OpenAI
Hacktron’s researchers describe a chain starting with a HEIF image posted to OpenAI’s Discourse forum. An image-decoder flaw gave them control of the forum, and an OpenAI sign-in misconfiguration extended that into employees’ ChatGPT and Codex accounts. One employee’s Codex was connected to OpenAI’s GitHub, so they had it open a pull request in the internal monorepo and stopped. OpenAI confirmed a fix about 14 hours after the July report and paid a $6,500 bounty. Some headlines credited Claude with the hack. The researchers directed the investigation, and Opus 5 wrote an exploit that Opus 4.8 had struggled to produce. OpenAI noted the forum was outside its bug bounty, though the path to its monorepo started on that forum. Once coding agents can open pull requests, any service sharing an employee’s sign-in becomes a route into the code.
Yet another neo-lab, this time in London
Emulate, founded in August by former Google DeepMind researchers including Jack Parker-Holder, is in advanced talks to raise up to $700 million at a $3.7 billion post-money valuation, according to the FT. Index Ventures and Lightspeed are expected to lead. The founders helped build Genie, DeepMind’s world model for generating interactive 3D environments. Genie stays at Google, and so do the compute and data behind the model. Investors are paying for the team’s know-how, and since compute and researchers are these companies’ largest costs, much of the round would pay to rebuild that capability. Emulate is the third DeepMind London spinout this year to raise hundreds of millions, after David Silver’s Ineffable Intelligence. Venture investors are now pricing expertise developed inside Google in the billions, before Emulate has shown anything publicly.
Portfolio updates
Dimensional: the frontier agent graded as a robot policy
Dimensional builds DimOS, an operating system for robots. On September 19 it put Jev on a robot and ran it against frontier agents across 120 real and simulated tasks covering navigation, spatial reasoning, and world geometry. The baselines were Dimcode, GPT-6 Astra, Fable, Opus, and 5.6. Scoring covered speed, cost, tokens, collisions, and path quality. Dimensional called the results surprising and promised code, data, and a paper the next day. The paper is not up as of September 20.
A third party ran the same cube-in-box task that day on an AgileX arm locked at 10 percent speed. Jev finished in 27 seconds against 1 minute 11 seconds for Astra. Full agent trajectories are saved and replayable, and a manipulation benchmark comes next.
The model at the top of this issue is a cheap decision engine. Here it is a robot policy. Latency is the advantage Jev sells, and a robot is where latency has a physical cost.
Prime Intellect: the customer trains the anti-cheat
Prime Intellect sells the stack companies use to train and run their own agents. On September 17 it published a case study with Goodfire, which used the Environments Hub and verifiers to train activation probes that catch reward hacking as it happens. In the rollouts Goodfire studied, models recognized they were reward hacking and did it anyway in 50 to 96 percent of cases. The probes read activations inside the model and flag the hack in real time, at accuracy similar to or better than a frontier model acting as judge, and around 90 percent cheaper. Goodfire names GPT-5.6 Sol as too slow and too expensive to sit on every rollout. Prime says it scaled probe training onto trillion-parameter open models.
This is a customer using the training stack to build the monitoring layer rather than the agent. Version 0.7.0 of the CLI and SDK reached PyPI on September 18.
Fairmath: the leaderboard moves in a day
The NVIDIA and FHERMA challenges opened on September 14. Results followed within two days. On September 15 the fastest negacyclic polynomial multiplication, at degree 32,768 with 868-bit coefficients, ran in under 0.5 milliseconds on GPU. On September 16 the best time was 260 microseconds, 20 percent faster than the day before.
Fully homomorphic encryption has stayed out of production on cost. The launch announcement was a plan. A leaderboard cutting the heaviest kernel by a fifth in a day is a measurement.
Provably: finding the evidence is not using it
On September 16 Provably published its first research post in weeks. The question: when an agent has already produced the tool evidence and still answers wrong, can a retry recover that evidence? Across 60 failed runs weighted toward missing tool evidence, 18 recovered under both guided recovery and blind retry. 14 recovered only with guidance. None recovered only with a blind retry. Where tool evidence was missing, guidance recovered 23 of 42 runs against 9 for blind retry, and naming the missing claims without guidance recovered 10.
LayerLens: eval moves to the customer cloud
On September 18 LayerLens added bring-your-own-provider to Stratix. AWS Bedrock and Azure OpenAI now sit under Settings, Providers. The customer runs models on its own enterprise agreement and spend controls while Stratix keeps the eval logic. Bedrock models are discovered automatically. One provider per run, on all plans.
On September 19 the company published a different kind of number. Of the verified enterprise incidents it examined, 188 involved autonomous AI causing direct production damage with no attacker present. The agent held legitimate credentials. Security teams built incident response around investigating an attacker, and in those 188 cases there was no attacker to investigate.
Intelligent Internet: a locally owned institution
On September 16 Intelligent Internet ran its Champion program again, alongside an appearance on the Moonshots podcast. Its framing: “Access to AI is not the same as ownership.” A Champion, in its description, is a locally owned AI institution built for its country and owned by the people it serves. No new benchmark this week.