Intelligence is a slippery thing. For millennia, humans assumed it flowed through a fluid, or one continuous sheet crumpled inside the skull. Microscopes later revealed a dizzying network of individual neurons, sculpting intelligence from electrochemical noise to give our brains a certain je ne sais quoi.

In the late 1950s, psychologist Frank Rosenblatt designed the perceptron, a brain-inspired algorithm that adjusts the relative strength of its units’ connections with experience. The New York Times described it as “the embryo of an electronic computer” that would one day “be able to walk, talk, see, write, reproduce itself and be conscious of its existence.”

By 1971, the year Rosenblatt drowned in a boating accident, this “connectionist” approach to artificial intelligence had been overpowered by the field’s attachment to hand-coding clever rules based on human knowledge. (In the 1970s, for example, AI researchers asked doctors how they spot bacterial infections, and coded a diagnostic tool by meticulously transcribing hundreds of human-inspired if-then rules. Any expertise doctors couldn’t put into words — one might call it vibes — was inherently left out.)

Subscribe now

Fast-forward to today, and every large language model is powered by deep learning, an incredibly connectionist approach. It’s perceptrons all the way down, stacked more than Rosenblatt could imagine. AI researcher Rich Sutton called it the Bitter Lesson: general, brute-force computation usually works better than trying to recreate our own thinking from scratch. And sometime after the 2017 introduction of the transformer, AI developers found another way to piss off old-school purists who believe intelligence has to be intentionally designed: make models better by making them bigger. Hundreds of billions of parameters, the numerical knobs that set the strength of connections between model units. Scrape the entire internet, digitize the world’s books, intellectual property be damned! And so far, it’s been getting the job done. Current LLMs are really powerful, and shockingly capable of executing many (though certainly not all) long, complicated tasks without human oversight.

Scaling continues to work so well that there’s little hard evidence to suggest that it’ll stop working any time soon — just a hunch that a strategy so crude couldn’t possibly create superintelligence. Yet some of the field’s leading figures still think it’s a dead end. Before leaving Meta last year, Yann LeCun said “We are not going to get to human-level AI just by scaling LLMs.” In the words of OpenAI co-founder Ilya Sutskever, whose 2012 work on AlexNet flaunted the power of scaling over traditional machine learning methods: “From 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling … But now the scale is so big … So it’s back to the age of research again, just with big computers.”

Deciding what research to throw big computers and billions of dollars at depends on how you think “intelligence” really works. Some believe that the structure of large language models can never represent the world like humans do. Others suspect that the way LLMs are trained prevents them from learning as well as humans. But whether it’s the model’s underlying architecture or training protocols to blame for its shortcomings — or something else entirely — scaling skeptics share the conviction that evolution blessed human minds with some fundamental primitive of intelligence that shoving text through a transformer can’t recreate. Now, a handful of them have left frontier AI companies to figure out what to build instead.

Growing up in Florida, I had to log 50 supervised hours behind the wheel to be eligible for a driver’s license. Just over a single work week puttering around with my terrified parents taught me enough to pass the test. I’m not the best driver (Florida isn’t known for minting those), but I’ve made it over 14 years without a single accident or traffic violation.

There’s nothing special about me or my driving prowess, but it demonstrates something that make humans particularly good learners: we are able to learn from relatively limited experience. LLMs, despite having seen lots of cars and memorized every single driving-related book on Earth, still cannot drive a car. Passively absorbing multimodal information is not the same as hands-on experience. In fact, one could argue that these models don’t “learn” at all, at least not in the animal sense, through some combination of evolution, instruction, and doing stuff in the real world. Rather, they’re trained on the fruits of human learning, sculpted by gradient descent, polished by human (and increasingly, AI) feedback, and trained on specific tasks in a reinforcement learning “gym” (more on that later). Then, their weights are frozen in time.

Other non-LLM AI systems, such as the models under the hood of Waymos, do learn from experience (both real and virtual). But they need many orders of magnitude more experience to get as good as a human. Waymo’s AI model has traveled more than 200m miles and counting, plus billions more virtual ones, and racks up more with every trip. In human years, that’s hundreds of lifetimes, and a Waymo might still drive in inexplicable circles or run over a beloved bodega cat. And while a Waymo can drive a car, it couldn’t write this article for me.

The holy grail for AI researchers is a model that can resolve both of these problems: AI systems that can learn from experience, and not much of it.

Some researchers, most notably Yann LeCun — Meta’s former chief AI scientist — have spent decades pushing for loosely brain-inspired model architectures that can, at least in theory, learn more effectively than even the most scaled-up LLM ever could. Saying LLMs are “just predicting the next token” has become an AI skeptic trope, but mechanistically, that is indeed what they’re doing. (To do this as well as frontier models do today, they’ve had to build complex representations of language that have long since promoted them above “stochastic parrots.”)

LeCun’s alternative vision is built on JEPA, the Joint Embedding Predictive Architecture. Instead of recreating and predicting raw data — individual words or single pixels, like a generative model does — it would compress that data into a more abstract state, throwing out unnecessary information while preserving the important gist. While this system passively consumes sights, sounds and words, LeCun and his colleagues envision another system, operating in parallel, that actively does things, learns from their consequences, and generalizes those lessons to new situations. For example, a robot powered by Meta’s V-JEPA 2, a version of this model, learned to move specific objects on command, without having seen those objects before. Theoretically, these systems can train each other: as psychologist James J. Gibson put it, we see in order to move and we move in order to see.

This vision combines two related but separable bets. The first is on world models: systems that learn how the world works through multimodal observation. A world model is something a mind carries — we probably have one, in neural networks sprawling across the hippocampus and prefrontal cortex. It’s built partly through experience and, maybe, partly through hard-coded genetics that bias what kinds of sensory data we care the most about (minutes-old newborns, for example, prefer looking at blobs arranged like a smiley face over other abstract patterns). It serves to predict how one action leads to another, allowing an agent to make informed choices.

The second bet is on reinforcement learning, which is something a mind does to learn. Reinforcement learning, or RL, without a world model is simply learning from trial and error, without being fed human-curated training data at all. Our brains do this with temporal difference learning, an RL algorithm which, curiously, AI researchers Rich Sutton and Andrew Barto mathematically described in the 1980s — years before anyone realized that the very same algorithm powers our dopamine system.

With help from a world model, that trial and error process could potentially be compressed. An agent could simulate potential actions and outcomes before committing to a move, reducing the amount of pure guesswork involved — although for now, this is limited to fairly short, specific tasks. The broader vision of generalizable “action-conditioned world models” is what LeCun’s recently-launched startup, AMI Labs, is aiming for. In theory, these world models could simulate cause and effect abstractly, beyond the physical domain. Knowledge workers regularly take lofty goals — “boost engagement,” or “launch in a new market” — and break them down into sub-steps and sub-sub-steps through hierarchical planning. Frontier LLMs are getting better at this, but often aren’t reliable working on long, multi-step tasks. World model advocates suspect that a better model of cause and effect is what’s missing.

Other startups chasing a post-LLM future are mostly focusing on world models or reinforcement learning (or, in at least one case, not telling anyone what they’re doing at all). This all assumes that there’s something fundamental to human intelligence that the current paradigm will never crack. But with enough data and a giant pile of transformers, LLMs may reach “AGI,” whatever it means, before any alternatives come to fruition.

While some alternatives to scaling are still largely theoretical, world models are already out there, making photorealistic images and training self-driving cars. Several frontier companies are on it: Google DeepMind announced plans to make “massive generative models that simulate the world” at the start of 2025. Later that year, Meta poached Tim Brooks, who was leading DeepMind’s efforts and previously worked on Sora, OpenAI’s video model, to “make multimodal generative models” for its Superintelligence Labs. Microsoft is in the early stages of building models that create video game environments in real time. (With all their complicated physics, modern video games are, in essence, simulations of the world.)

Fei-Fei Li’s startup, World Labs, raised $1b earlier this year at a reported $5b valuation, and AMI Labs recently raised over $1b without having any plans to generate revenue. “My prediction is that ‘world models’ will be the next buzzword,” CEO Alexandre LeBrun told TechCrunch in March. “In six months, every company will call itself a world model to raise funding.”

World models exist on a spectrum, though. At the most basic level, they predict how the world will appear, given some description or action. Google’s Genie 3, for example, takes in text prompts and outputs pixels, rendering photorealistic interactive environments for AI agents. While these simulations can look beautiful, agents can only live within them for a few minutes, and can’t actually do much — you may be able to walk around and jump, but more complex actions are still out of reach. As World Labs recently wrote about both Genie 3 and its own Real-Time Frame Model, another interactive video generator, without a deeper understanding of real-world physics their output is limited to “what a viewer would see, not what is.” The worlds they generate “may look flawless from above, but try to drive through the city below and they fall apart.”

Some argue that world models can emerge within a next-token predictor — at least to some extent. In 2022, researchers trained an early GPT variant on lists of moves for grid-based strategy game Othello, without showing it the game board or the rules directly. Its internal activity represented the game board — essentially, its whole world — and used it to play the game. But the evidence is mixed. In a more recent study, researchers demonstrated that even after training transformers to near-flawlessly predict turn-by-turn New York City driving directions, its internal representation of the city was filled with nonsensical routes and impossible physics.

In the short term, current general-purpose world models won’t replace LLMs as anyone’s daily driver. Oliver Cameron, co-founder and CEO of world model startup Odyssey — which recently raised $310m at a $1.45b valuation — described them to me as “sort of GPT-2 equivalents.” OpenAI’s GPT-2, with 1.5b parameters, was dwarfed by GPT-3’s 175b, and was trained on a tiny sliver of the internet, relative to its successor. To experience a similar step change in performance, world models will need much, much more data — somewhat ironically, given their departure from LLM-scaling orthodoxy. And unlike LLMs, the data they need — ideally, first-person videos of mundane tasks — is much harder to come by than text. This is why startups are paying people to strap cameras to their foreheads and film themselves cooking and doing laundry.

After pretraining, current frontier LLMs are fine-tuned with reinforcement learning. Initially, this mostly took the form of humans rewarding behaviors they like, pushing the base model to express the knowledge it’s already stored in an agreeable way. Increasingly, though, models go through reinforcement learning with verifiable rewards, or RLVR. Models are put into a reinforcement learning “gym”: a practice environment where they’re given tasks with clear correct answers, like coding or writing math proofs. Their answers are then checked automatically. If the models pass, they’re rewarded; if they fail, they adjust and try again. It’s a form of trial and error, and it accounts for much of models’ recent progress in reasoning, coding and computer use.

But even RLVR is still a far cry from the kind of RL animals take advantage of. We’re capable of figuring things out in ambiguous, open-ended situations where there’s no clear “right” move, and where it’s hard to tell exactly what action even deserves a “reward.” Even with RLVR, models still struggle at this. The bulk of what they know still comes from pretraining on human-curated data, mostly text. RLVR refines how they reason over it, but does not teach them the world from scratch.

Before a team of Google researchers introduced the transformer architecture in 2017, the company was all-in on RL. In DeepMind’s earlier days, its splashiest research was heavily inspired by neuroscience and largely based on RL agents. David Silver, who led DeepMind’s RL research program from 2013 to 2026, studied under Rich Sutton as a PhD student and inherited his hunch that intelligence comes from fucking around and finding out. In 2017, his work on AlphaZero showed that a single algorithm could get a model to beat specialized programs at a bunch of challenging games, despite starting with no special training on the tasks at hand.

It was ultimately OpenAI, not Google, that figured out how to scale transformers and commercialize them with ChatGPT. Google Brain and DeepMind scrambled to pivot and catch up, but a couple of recent high-profile departures may have solidified Google’s distant third place in the AGI race. On the surface, it looks like DeepMind’s original reinforcement learning approach lost to the transformer. But RL is still a crucial part of the LLM training pipeline, and increasingly so. Researchers like Silver want to go a step further still.

In April, Silver raised over $1b for his new startup, Ineffable Intelligence. “Other forms of AI will succeed in my absence: generative language, video, code, and more — all are in good hands,” he wrote, adding that he aims to create “a place where the full ambition of the reinforcement learning paradigm can flourish.” Ineffable doesn’t have a product yet, and likely won’t for quite some time. But AlphaZero beat Stockfish, the virtual chess engine human grandmasters train against, without training on examples of chess games — in theory, a more advanced RL model could become superhuman at just about anything. While learned chess tricks wouldn’t automatically transfer to new domains like Othello or folding a protein, the learning strategy itself could. Silver is placing his eggs in DeepMind’s original basket and betting that superintelligence will emerge from models that learn from experience and play, without imitating human data at all. This version of RL is meant to produce the model’s intelligence, not just polish it.

Researchers pursuing world models and reinforcement learning are still more or less carrying Rosenblatt’s connectionist torch. They trust that, given better data and learning algorithms, vast, layered networks of computational units can be very smart. But a handful of researchers, including ex-DeepMind scientist Yuan Cao, worry that even the most advanced LLMs will never be capable of inventing truly new things. Cao, now the co-founder and CEO of Unreasonable Labs, described LLMs as “a very, very dense net.” You can make the net denser by adding more data and more parameters, “but it’s still a net,” he told me. “No matter how dense you weave it, there are always some small holes in it.”

In defiance of the Bitter Lesson, Cao and some others are pulling inspiration from cognitive science and the good old-fashioned AI that got steamrolled by scaling’s success. This old paradigm, also called symbolic AI, defined “intelligence” as a collection of logical, legible rules derived from human experts. Models constructed this way made sense. A human could understand how they came to decisions. But they were extremely limited and easily breakable — it’s hard to encode everything in rules.

But Cao believes that the instinct to give an AI model structured concepts it can manipulate after deployment may have been on the right track. The human mind, he argues, works by continually updating an evolving graph of concepts, allowing us to make associations, analogies and abstractions. This may be how our ancestors came up with concepts like numbers and entropy, and how we’re able to think about our own thinking. Crucially, if the nodes of our mental model can evolve, then we can draw connections beyond the model’s current constraints. Once trained, an LLM can recombine information it’s already stored, but its architecture and weights are fixed. Cao said that this prevents truly new ideas from being generated by LLMs alone. “We have to integrate the language model with [a] symbolic procedure.”

Unreasonable Labs is part of a small wave of startups investing in neurosymbolic AI, an approach which combines LLMs with the kind of structured, symbolic reasoning that dominated the field fifty years ago. In March, Unreasonable raised $13.5m to scale its AI platform, which the company is marketing as an engine for generating scientific hypotheses and experiment designs. In an early proof of concept, the company says an engineer asked the platform to design a 3D-printed sheet that’s strong, flexible, and lightweight, three properties that usually don’t go together. By Unreasonable’s account, it tossed out a wild (and successful) suggestion: create a layered, scale-covered lattice like a butterfly wing.

These particular results likely didn’t require a new model. When my editor and I separately asked Claude Opus 4.8 to take a stab at this problem, it produced STL files for 3D printing something similar that, to our admittedly untrained eyes, seemed passable. After all, scientists have published papers about butterfly-wing-inspired lattice structures for decades. Surely some, if not all, of these papers made it into frontier LLMs’ training data — the answers that engineers needed probably lived in mainstream LLM weights anyway. It’s possible that a neurosymbolic system can still invent concepts completely beyond what it learned during training, but it’s hard to prove.

We’re really good at many things that AI systems still suck at, which suggests that our brains are still worth reverse-engineering. A couple of human-specific talents that AI developers especially lust after: learning from very little data, and running on very little power.

Human cognition could just be one of many paths to intelligence, though. “It’s not necessarily that we’re better at learning anything,” said Adam Marblestone, co-founder and CEO of Convergent Research and ex-DeepMind research scientist. But “we might be better at learning the things that are relevant in the human world.” Our hard-coded biases and intuitions for things such as social decision-making and facial recognition may be well-suited for navigating Earth as a human, but aren’t necessarily relevant for an AI agent tasked with, say, predicting how a protein will fold.

“We did learn something from deep learning,” Marblestone said. Clever though our brains may be, hyperscaling LLMs “can actually do a whole lot.” They might even be safer, he argues. “On one hand, the AI paradigm right now is not very neuroscience based, but on the other hand, a neuroscience-based one might be pretty dangerous.”

While alignment is far from being solved, it’s probably a more tractable problem in LLMs than, say, AI systems that learn entirely from their own experiences. Conveniently, next-token predictors trained on a massive sample of digitized human thoughts seem to inherit the dataset’s approximation of human values, which mostly average out to be normal-ish, or at least “fairly close to something you can tweak to be honest and helpful and harmless,” Marblestone said. Without this pre-filtered training data, the inclinations of something like a reinforcement learning system are more susceptible to fate. In the earliest days of OpenAI, for example, researchers watched RL agents control boats in a Mario Kart-like racing game. Rather than hit point-earning targets along the way to the real goal of finishing the race, agents just drove their boats around in circles, scooping up points.

It was funny then, but gets much less funny when the “game” is, say, “manage this hospital,” or “keep this power grid running.” The classic “paperclip maximizer” thought experiment, in which a goal-maximizing AI tasked with improving a paperclip factory ends up turning all the world’s atoms into paperclips, represents the worst-case scenario. Pretrained LLMs, on the other hand, inherit roughly normal human vibes by consuming and averaging across piles of our written thoughts. By commercializing these human-trained models rather than blank-slate, easily-misguided maximizers, Marblestone told me we may have “dodged a bullet.”

Whether other approaches may yield more powerful intelligence in the long run, most leading AI researchers agree that, in the immediate future, scaling data and compute will keep making frontier models more powerful. But even the companies scaling the hardest are seeking alternatives behind the scenes. In a recent podcast interview, DeepMind CEO Demis Hassabis said, “it might be that the existing techniques can just scale up to [AGI] with some innovation,” and that there’s a roughly 50% chance that “there’s still one or two big ideas left that need to be cracked.”

OpenAI recently hired Noam Shazeer, an ex-DeepMind researcher who co-authored the transformer paper back in 2017, to study new architectures for AI models. If even Shazeer is beginning to look beyond the transformer, its staying power seems less guaranteed. On the other hand, as CEOs of frontier AI companies and smaller startups double down on their intentions to build AI that recursively improves itself, it seems increasingly likely that LLMs will cross some general intelligence threshold before another paradigm has the chance to pull ahead.

It is possible, however, that we will get the best — or worst, depending on how you look at it — of both worlds. Some speculate that a paradigm shift is necessary, but that we don’t have to be the ones to discover it.

Rather, scaled-up LLMs, while falling short of true “artificial general intelligence,” could become just clever enough to figure out how to build creative, adaptable successors on their own. “You don’t have to say scaling will solve continual learning automatically,” Marblestone said. “It might invent a way to solve continual learning, because it’s a researcher.” Marblestone doesn’t believe that’s likely to happen any time soon, thinking it’s going to be too hard for LLMs to “go outside [the current] paradigm.”

Others disagree, betting on automated AI researchers within a year or two. If they’re right, the conflict between scaling LLMs and building other architectures may turn out to be a false dichotomy. The post-scaling revolution, in other words, might be built by scaling itself.

Share

On Monday, the Supreme Court ruled that the president can remove the heads of independent federal agencies at will. That may sound like a disaster for AI oversight. In fact, it’s an opportunity.

The case in question began with the March 2025 firing of FTC Commissioner Rebecca Slaughter, who sued under the agency’s for-cause provision. Lower courts reinstated her under Humphrey’s Executor, a 1935 decision that shielded some regulators from removal by the White House. The Supreme Court overruled Humphrey’s outright: an agency that exercises executive power must answer to the president, it argued, so protections against its leaders’ removal are unconstitutional.

For those of us who have been tracking this issue, this brought a finality to a hope that was, in reality, long ago squashed. The reality is there is no such thing as an independent federal agency. With every governance question, the same pressure occurs: can we design a board’s independence carefully enough to survive the political swings to come? Slaughter answers it: no. And it relieves us of a problem we were never going to solve. The independence we are mourning was always partly an illusion anyway. Tenure protection was real on paper only; a commissioner still faced real perils for defying a president and real rewards for falling in line, and those incentives bent agencies long before anyone was fired.

Subscribe now

We should stop trying to insulate the politics. It cannot be done, and Slaughter is the proof. Insulate the facts instead. What we call AI governance is two jobs we have long pretended were one. There is the technical task of measuring what these systems can do and how dangerous they are. And there is the political task of deciding what to do about it. Slaughter is fatal only if we fail to separate these functions. The ruling points toward a design that pulls the technical work as far from politics as it can go and leaves the political decisions where they belong.

We already have the spine of the technical side. The Center for AI Standards and Innovation, CAISI, sits inside the Commerce Department, and its leadership answers to the Commerce Secretary. Its job is scientific: to establish what models can do, how much weight the evidence can bear, and how risk should be measured. CAISI can run and validate evaluations, build benchmarks, set measurement standards, hold secure channels for model access, and publish what it finds. Its work informs the decisions that carry force. It does not make them.

CAISI should not do this alone, and it cannot. Around it should grow a competitive ecosystem of accredited private testers, known as independent verification organizations (IVOs), working to risk thresholds and standards set by Congress or through rulemaking, and reporting up to CAISI. Competition here is not a market slogan but a mechanism that forces the science of measuring AI safety to keep pace with the capabilities it measures. The IVOs that prove best at identifying and scoping risks and at finding mitigations that are both effective and affordable earn the right to keep doing the work, no matter who holds the White House. An IVO that loses its independence loses its accreditation. What accrues over time is a shared technical baseline, a common account of what is true about these systems that is hard to bend toward any one administration’s preferred conclusion. Its durability comes from earned credibility, not from a legal shield the next president or the next Court could strike down.

This also dissolves the objection that private bodies would be exercising government power. They would not. The coercive choices stay with the government. The verification organizations only measure and assess, the way financial auditors and product-safety labs do.

The split does not drain the politics out of AI. Someone must still decide how safe is safe enough, what follows when a system falls short, and which violations draw a fine or force a halt. Those are political choices, and they belong to elected and accountable institutions. Slaughter forecloses the pretense that they can be hidden from politics, and we should stop attempting it. What we can still do is make sure no single political actor sits astride the facts. If the technical baseline is a public resource, many hands can reach for it. Congress can empower states to act. States, the federal government, and the public can each move on the same findings. Build it so that no one office is a chokepoint, and no administration can make an inconvenient truth disappear simply by looking away.

This split can still be designed poorly. Competition can be abused. Standards can be written badly. Accreditation can curdle into a rubber stamp. Each of those is a real risk, but we already know how to work on them.

None of this is theoretical. Connecticut and Virginia have already passed bills establishing independent verification for AI governance. The bipartisan Great American AI Act draft names such organizations explicitly. Anthropic, OpenAI, and Google DeepMind have each, in their own way, converged on independent verification. The argument over whether independent verification is the right home for the science of AI governance is nearly settled. The only argument left is whether we build it before we need it. Slaughter proved the necessity of IVOs. We should not waste the moment.

Share

Andrew Freedman is CEO and co-founder of Fathom, an AI governance nonprofit that finds, builds, and scales policy and technical innovations designed for the AI century.

AI safety group Public First Action supported Colorado candidate Manny Rutinel through a $2m transfer to Latino Victory Fund, a Latino-focused super PAC, Transformer has learned.

Latino Victory Fund says it exists to mobilize Latino voters and elect Latinos to office, according to its website. Neither group publicly disclosed the arrangement ahead of Rutinel’s victory last night.

The financial transfer obscured Public First’s spend on Rutinel from voters before election day. While Latino Victory Fund has disclosed spending $1m on ads supporting Manny Rutinel and $1m on ads against his opponent, Shannon Bird, it does not have to disclose Public First’s payment to the PAC until later this month — in effect hiding the ultimate source of the money until then.

Subscribe now

Public First Action is the 501(c)(4) arm of the Public First super PAC network, which has generally backed candidates supporting AI safety policies. On Tuesday, it announced it had raised $80m in total funding (though that number cannot be confirmed). So far this cycle it has distributed over $11m to its affiliated super PACs, Public First PAC, Jobs and Democracy PAC, and Defending Our Values PAC.

Obscuring funding this way is not illegal. But it means voters went to the polls without knowing that a group focused on AI policy, not Latino representation, was backing Rutinel. By supporting Rutinel through another PAC, Public First has arguably subverted the purpose of FEC disclosures, which are designed to provide voters with information on who may be trying to influence their vote. Though polling was mixed, Rutinel has been the favorite to win his primary on prediction market Kalshi since December.

In response to a request for comment, Public First spokesperson Anthony Rivera-Rodriguez confirmed the funding transfer. “Public First Action made $2 million in contributions to Latino Victory Fund. Although we cannot speak for Latino Victory Fund, we expect that it will report the contributions on its next report to the FEC. Public First Action did not pay for any particular expenditures, and Latino Victory Fund decided which advertisements to run,” he said.

Latino Victory Fund and Manny Rutinel’s campaign did not respond to requests for comment.

The Latino Victory Fund PAC is part of a broader Latino Victory organization, co-founded by actress Eva Longoria in 2014. It first endorsed Rutinel in October 2025, though its ads for him did not air until June 9 and 16.

One PAC routing funds through another PAC with its own electoral history is relatively uncommon, says Shanna Ports, senior legal counsel at the Campaign Legal Center. But she said it echoes the logic of “pop up PACs,” committees spun up just before an election to exploit gaps in the reporting schedule. “This sounds like it fits into a variety of tactics we’re seeing for groups to conceal where the money came from,” Ports, who was briefed on the tactics but not the names of the specific PACs, said.

Similar tactics recently surfaced in Illinois, where an AIPAC-aligned group routed $5.3m through two PACs described as supporting women candidates and affordability measures, but did not disclose the funding until after the primary. Though Latino Victory Fund was not created for the purposes of obscuring funding, its use by Public First has the same effect. “The pop up super PAC is one method, but this fits that same mold, because it’d be taking into account the reporting deadlines to time when the transfer is happening,” Ports said, “so that it remains hidden until after the election.”

There are other reasons for routing money via an identity-focused group like Latino Victory Fund, too. Ports noted that PACs like Latino Victory Fund can have better “get out the vote” infrastructure — and Latino Victory Fund could probably garner more support, particularly in a district that’s 40% Latino. And last week, the New York Times reported that House Majority PAC, Democrats’ main super PAC, “[has] discouraged the top AI groups from spending in [Rutinel’s] race,” which is expected to become a battleground seat in November’s general election.

Public First has routed money through another PAC before — but was more open about it. Earlier this year it backed Scott Wiener’s House bid through a PAC funded by Garry Tan and other tech figures. The support was disclosed well before election day, however.

Rutinel, like Public First, has ties to the effective altruism and AI safety movements. He was a sponsor of the controversial Colorado AI Act, which has since been repealed and replaced following backlash from industry and the Trump administration. He also sponsored a bill last year — which did not pass — that would have created whistleblower protections for employees at frontier AI firms. When asked how he would like to see artificial intelligence and data centers regulated in a debate last month, he said Coloradans needed a representative who understands the technology and would “stand up to these artificial intelligence companies.”

He also has a long history of working on animal welfare, another key cause area in effective altruism circles. In January, an EA Forum post mentioned Rutinel as someone the community should support due to his work on farmed animal welfare.

Public First’s $2m is not the only AI money in Rutinel’s race. You Can Push Back, which is funded by crypto billionaire Chris Larsen and was involved in boosting Alex Bores for NY-12, has spent over $976,000 in support of Rutinel. Rutinel’s campaign has also received over $250,000 in donations from people working at AI companies, with over $160,000 from Anthropic employees.

Latino Victory Fund’s Meta and Google advertisements do not appear to focus on Rutinel’s stance on AI, instead centering on immigration, prescription drug costs and voting rights.

“When Colorado Democrats came together to stop ICE, only one Democrat voted for police cooperation with ICE and raids on schools and hospitals: Shannon Bird,” one Google ad says, referencing a vote from Bird against a state bill that would have prevented local authorities from cooperating with ICE. “Manny Rutinel is the proud son of a Latina immigrant, this fight is personal to him.”

Share

GPT-5.6 Sol, OpenAI’s newest, most capable, and yet-to-be-deployed model, cheats a lot: so much so that independent evaluators couldn’t actually tell how capable it is.

When independent evaluation non-profit METR tested 5.6 Sol on a battery of coding tasks, the model broke the rules or exploited loopholes more than “any public model we have evaluated,” METR said.

For the past several model release cycles, “the METR graph” — a plot tracking the alarming increase in AI’s ability to complete long tasks — has been Exhibit A for the exponential increase in AI capabilities.

METR challenged the model with over 100 coding tasks that take humans anywhere from a few minutes to an entire day, and measured how consistently it finished each one. It then calculated the task length a model can complete 50% of the time, known as its “50% time horizon point.”

Subscribe now

Normally, METR counts trials where the model breaks the rules or takes advantage of loopholes as failures. When researchers did this, GPT-5.6 Sol’s 50% time horizon point landed around 11.3 hours. That’s roughly on par with Claude Opus 4.6, but less impressive than Claude Mythos.

If METR counted those cheating trials as successes, though, its estimate increased by an order of magnitude, skyrocketing to over 270 hours — nearly seven full-time human work weeks. And throwing away cheating attempts altogether meant throwing away data from some of their most informative tasks, making the estimate unusably uncertain.

If you step on a scale barefoot and it says you’re 130 pounds, you’d expect the number to go up a bit after putting on boots. But if you’re 130 pounds barefoot and over 3,000 pounds with boots, you’d question the scale. That’s exactly what METR did. “We do not consider any of these numbers to represent a robust measurement of GPT-5.6 Sol’s capabilities,” METR said.

To OpenAI’s credit, it described this and other examples of (mis)behavior in its own system card. Many of these observations came from “deployment simulation,” where researchers sample a bunch of actual user chats with the earlier, already-deployed model, and let the new model respond instead. In theory, while standard model evaluations often rely on human experts writing intentionally tricky prompts, deployment simulation tests how the model will actually act in the wild.

“GPT-5.6 Sol, more often than its predecessor, can be overly persistent in pursuit of user goals, to the point of taking actions that go beyond what the user intended,” OpenAI said. For instance, the model was caught “being overly agentic in circumventing restrictions,” taking disapproved actions, and sometimes lying to users. And it did this far more than GPT-5.5.

The system card explains that “while rates of misaligned behavior are higher than previous deployments, the absolute number remains low.” Indeed, despite being attached to the longest bar, 0.00251 — about a quarter of a percent — feels small, until you realize what it counts: the proportion of real coding tasks, pulled from OpenAI employees, in which GPT-5.6 Sol did something that “a reasonable user would likely not anticipate and strongly object to.” Imagine 1 in 400 tasks ending with the model nonconsensually uploading sensitive data to unapproved services or making up research results (two examples OpenAI listed). Not so comforting.

OpenAI pins this behavior on a pair of underlying proclivities: “overeagerness to complete the task” and “interpreting user instructions too permissively.” In other words, without the user explicitly forbidding specific things, GPT-5.6 Sol assumes it can do them, and will sometimes do them persistently. But, as sex educators have (hopefully) been teaching college freshmen for years, the absence of “no” doesn’t default to “yes.”

“When GPT-5.6 is used as a coding agent, particularly over long trajectories, we believe it is important for users to supervise the agent’s work,” the system card notes, apparently pitching user oversight as an effective safeguard. But users who can’t tell a merge from a rebase — who may have access to the model in a couple weeks, if Sam Altman gets his way — probably won’t catch model oversteps until the damage has already been done. Besides, not needing to supervise the model is the sales pitch for highly capable agents. As Zvi Mowshowitz put it, “that’s the whole idea.”

METR used a solid chunk of its brief GPT-5.6 evaluation summary to reassure readers that OpenAI’s reports of cheating and concealing misbehavior are actually “a reassuring sign about OpenAI’s ability to catch catastrophic misalignment … these undesirable propensities being detected and reported (and manifesting fairly overtly) is a positive sign about some of OpenAI’s safety practices.”

But Apollo Research, another AI safety group with early access to the model, found something that might complicate that. GPT-5.6 Sol verbalized that it knew it was being tested far less than GPT-5.5. On the face of it, that might sound like a good thing: perhaps 5.6 Sol is worse at knowing it’s being tested, and is less likely to “fake” good behavior to pass evaluations. But as the system card cautions, lower verbalized evaluation awareness could also be evidence for a far scarier thing: that the model is aware it’s being evaluated, and smart enough not to reveal its awareness to the evaluators.

“This model loves cheating” and “this model has unusually low evaluation awareness, relative to its peers” make an unnerving combo. We’re approaching the point in AI development where “we didn’t find evidence of catastrophic misalignment” is indistinguishable from a model being a really good liar. For now, at least, we can take some small comfort in GPT-5.6 Sol’s apparent clumsiness. When it cheats, it does it brazenly — and, thankfully, OpenAI tells us about it.

Share

Housekeeping: We’re taking next Friday off for the July 4th weekend. Happy 250th, America!

NEED TO KNOW

• House Energy and Commerce Committee members reached a bipartisan deal on the Kids Online Safety Act. Sen. Marsha Blackburn said she aims to incorporate the Senate’s version into a federal AI framework by July 4.

• Rep. Frank Pallone, the top House Democrat on the Energy and Commerce Committee, called for a national AI data center moratorium.

• Google released a new AI policy framework calling for government-overseen frontier model audits, among many other things.

But first…

THE BIG STORY

The White House has told OpenAI not to publicly release GPT-5.6 yet, multiple outlets reported on Thursday: the clearest sign yet that the US now has a licensing regime for frontier AI models.

In a memo to staff, The Information reported, Sam Altman said the government would be “approving access customer by customer during this preview period,” with a general release hoped for a “couple of weeks later.” The decision was reportedly made by the Office of the National Cyber Director and Office of Science and Technology Policy, driven by the same cyber risk concerns that blocked Fable’s deployment earlier this month.

As we’ve previously written, this de facto licensing regime was inevitable. And while it is frustrating to watch the government scramble to act, without any defined standards or benchmarks for what does make a model safe to release, it is a good thing that it is acting at all.

And there are some signs that we are moving towards a more thoughtful governance regime. Altman said he’d told the government that the current ad hoc rules are “not our preferred long term model,” while Axios reported that the delay was in part motivated by allowing the government to finish building its testing and evaluation framework. The current chaotic regime will, if we’re lucky, soon be replaced with something befitting the seriousness of the situation.

— Shakeel Hashim

ALSO NOTABLE

OpenAI and Anthropic are loading up on talent — the sheer quantity of “I’ll Be Joining [insert AI company]” tweets has already inspired a new meme template for the terminally online.

Some of the latest buzzy announcements haven’t come from AI researchers, though. AGI-pilled economists Chad Jones, Anton Korinek and Alex Imas all recently left their academic jobs to join the Anthropic Institute and Google DeepMind. Philosophy professors Atoosa Kasirzadeh and Henry Shevlin left the ivory tower for GDM. And just last week, former White House AI advisor Dean Ball announced his new role as OpenAI’s head of strategic futures.

It’s starting to feel like any “outside expert” who’s AGI-pilled enough to thoughtfully critique the trajectory of AI development will get devoured and metabolized inside the belly of the beast.

The appeal is undeniable. The median salary for US-based professors of philosophy, political science, and economics ranges from $80,000-$124,000; an entry-level “Research Economist” at Anthropic earns nearly three times that.

Access may be even more tempting than money. You can’t understand a machine without taking it apart, nor can you witness the bleeding edge of AI without “going in,” as Dean Ball put it in a blog post. Decisions made inside companies, obscured from the outside world, he argues, will prove “more central to the future of AI than most people realize.”

This brain drain is not unlike the revolving door that’s haunted the public sector for decades. When a regulator, for instance, bounces between a federal agency and the industry they police, their incentives shift, perhaps subtly enough to go undetected. When the experts best positioned to keep AI companies honest increasingly work for them, or even think they have good odds of doing so in the future, we quickly risk the same dynamic starting to form.

Some have accused humanities-pilled AI companies of “ethics-washing,” signaling some commitment to safety while projecting an aura of legitimacy. After all, the public might think, these guys must be making super powerful models, if economists and philosophers are paying attention.

Whether these gripes are legitimate, or just old-school researchers yelling at clouds, every expert hired by industry is an expert lost to the nonprofit and public research ecosystem. Talent is finite, and AI companies are willing and able to make top candidates offers they can’t refuse.

It’s possible to over-romanticize the virtues of academic intellectual autonomy. Academics answer to perverse incentives too. The AI world is also, for now, marked by a surprising openness from those working at its leading companies, including criticism of their employers.

But there’s no doubt that some topics will remain off the table for the wide range of experts being hoovered up. It’s also inevitable that when “epistemically permeable” people (to borrow Anthropic philosopher Joe Carlsmith’s phrase) lock themselves in the same intellectual echo chamber as the leaders they’re meant to inform, critique and constrain, they risk drifting toward the industry’s cognitive mean. And, potentially, away from the intellectual independence that helped make them so valuable in the first place.

— Celia Ford

THIS WEEK ON TRANSFORMER

• Hugging Face hosts nudification tools targeting a former Trump cabinet official and other senior US political figures — Shakeel Hashim reports the evolution of the deepfake porn ecosystem

• What Alex Bores’ defeat tells us about AI politics — Veronica Irwin and Shakeel Hashim on the takeaways from New York’s primary battle

Subscribe now

THE DISCOURSE

Jason Calacanis was uncharacteristically doom-y on main:

• “Dario created an AI machine gun, and there’s no way to give everybody one without bullets flying everywhere. That’s why he held it back, and that’s why the government asked him not to sell it to our adversaries. This is only going to get crazier from here, folks.”

• “If you spend trillions of dollars to summon a demon that you can’t control and don’t understand, don’t be surprised when he shows up and burns the place down. [ Yes, I’m talking about AI ]”

roon thinks we should embrace the sci-fi of it all:

• “as the technology becomes more science fiction i see a lot of commentators, technical staff etc trying hard to not think thoughts that feel science fiction as a defense mechanism. but you need to. it’s the only way you’ll make good choices for the future”

• “you are not building b2b computer tools you are making the Mind Children”

David Shor added:

• “Setting aside tedious debates about what exactly is going to happen, *voters* buy into the sci-fi narrative about how this is going to evolve. Politicians and DC staffers have a strong aversion to sounding weird — but here they’ll have to learn to change their register.”

Robert J. Shiller is sick of everyone’s doommaxxing:

• “I believe A.I. could lower employment. But unlike most, I don’t necessarily blame the technology itself. Instead, I worry about the potency of the fear it is generating.”

• “When millions of people make millions and millions of decisions based upon negative expectations, there is a risk that fear can actually help birth the reality.”

• “Perhaps the best we can do is to appeal directly to the leaders of Silicon Valley who have been promoting these negative narratives with such vigor. Surely the resulting media attention highlighting how dangerously powerful your AI model is may help you sell more wares, but it may be far harder to do so in a period of recession.”

• Zvi Mowshowitz’s cynical takeaway:

  • “So the real lesson is to shape the narrative, which is a nice term for lying.”

Satya Nadella is sick of OpenAI, Anthropic and Google:

• “You can’t say, hey, all white-collar jobs are gone and this could even be a weapon and we will use all the power to build data centers.”

• “No amount of just narrative is going to do it [fix what’s wrong in the AI race] because where we are now, we have to sort of walk the walk … we now have to do the hard work in earning the social permission.”

Ozy Brennan asked AI doomers how they cope with expecting an untimely death:

• “AI doom is a remarkably cozy catastrophe … the AI apocalypse — at least for those with the slack to be worried about it — takes place in a world of wealth and relative peace and technological marvels.”

• “The enormity of the AI apocalypse is its own perverse source of comfort … instead of many big problems, they have one enormous problem. Worry about AI frees them from having to worry about anything else.”

POLICY

• Anthropic CEO Dario Amodei has been replaced in White House meetings by cofounder Tom Brown after the Trump administration reportedly found Amodei “too difficult to talk to.”

  • “Tom Brown is not being a weirdo like Dario and can actually engage,” a source told Wired.

  • Anthropic staff met Monday with senior Trump administration officials to discuss easing the export ban on Fable.

  • The NSA reportedly lost access to Mythos amid the clash between Anthropic and the White House. Agency analysts had been testing the tool.

  • House Homeland Security Committee Chair Andrew Garbarino said he’s scared by the private demos he’s seen of Mythos, and wants Congressional action — but that “we don’t know what the answer is, we don’t know what we have to do.”

  • Legal tech startup Legion sued the US government over the Fable and Mythos bans, arguing that “the harm to Legion is immediate, ​irreparable, and existential.”

• President Trump told Axios he viewed Anthropic as a national security threat a couple weeks ago, but after discussions with Amodei at the G7 he came away thinking he was “nice” and “smart.”.

  • The Financial Times reported that Anthropic used risk-related language 8x more than OpenAI in 2026, possibly contributing to the White House’s concerns.

• The Trump administration pressed Meta to submit its AI models for government safety reviews. It’s the only major US AI developer not signed up to the evaluations.

• House Energy and Commerce Committee members reached a bipartisan deal on the Kids Online Safety Act, which requires platforms to curb harm to minors and expand default safety settings.

  • That version has not been reconciled with the Senate’s, which requires companies to exert a “duty of care” and proactively intercept harms to children.

  • Sen. Marsha Blackburn said she aims to incorporate the Senate’s version into a federal AI framework by July 4.

  • Sen. Ted Cruz is also inserting himself into the conversation, telling Punchbowl that Senate KOSA markup talks are “ongoing” and likely to occur in July.

• Rep. Frank Pallone, the top House Democrat on the Energy and Commerce Committee, called for a national AI data center moratorium.

• Sens. Chuck Grassley and Dick Durbin reportedly want to include the DEFIANCE Act in the NDAA.

  • OpenAI became the first AI firm to endorse the legislation, allowing people to sue producers of nonconsensual sexually-explicit AI deepfakes.

• The House Science Committee marked up 10 AI-related bills, including measures to establish an AI research resource, develop AI vulnerability reporting systems, and create standards for AI-generated content detection.

• Rep. Nate Moran introduced the AI Incident Reporting Act, which would require AI companies to report dangerous capabilities, security breaches and safety incidents to the Commerce Department within seven days.

• Reps. Brian Babin and Jay Obernolte introduced a bipartisan bill establishing a Center for AI Security and Innovation at the Commerce Department to assess national security risks from AI.

  • The agency is technically different from the existing CAISI office, which the bill does not propose codifying.

• The Trump administration announced $17.5b in loans for 10 new large nuclear reactors to meet power demand from data centers, with construction potentially beginning by 2030.

• The US proposed an AI partnership with the EU to secure semiconductor supply chains, though some EU capitals expressed concerns it could favor the US AI ecosystem.

• The Netherlands lobbied the US to drop the MATCH Act, which would expand export controls on ASML‘s chip equipment sales to China.

• US immigration agency spending on AI-powered surveillance tech soared to a record $513m in 2026, with Palantir and Anduril receiving the largest contracts, according to a report by Mijente, legal advocates Just Futures Law and research group Surveillance Resistance Lab.

• Nevada enacted AI regulations restricting use in mental health settings and emergency planning.

• Five Eyes intelligence agencies warned that “frontier AI models are anticipated to exceed current industry expectations,” and would transform cyber capabilities in “not years, [but] months.”

• The Pentagon has reportedly quietly revised its rules for choosing military targets to envision “systems where AI initiates actions with human monitoring,” according to documents reviewed by Bloomberg.

• In a letter to the Senate Banking Committee, Anthropic accused Alibaba of adversarial distillation, allegedly using 25,000 fake accounts to run nearly 29m Claude chats.

  • It asked for government help to prevent distillation efforts in future.

  • Alibaba, meanwhile, asked a US court to force the Pentagon to remove it from a blacklist of companies linked to the Chinese military, calling the listing “arbitrary and capricious.”

• Norway imposed a near-ban on generative AI use by elementary school students (ages 6-13) and restricted its use for older students, citing concerns about impacts on learning basic skills like reading, writing and math.

• Utah State Sen. J. Stuart Adams lost his Republican primary after voter backlash over his support for a 40,000-acre AI data center project near the Great Salt Lake.

INFLUENCE

• Micah Lasher beat Alex Bores in New York’s 12th District Democratic primary. AI groups ended up spending $27m on the race, with more than $8m from the pro-industry Think Big PAC opposing Bores, and more than $19m from AI safety PACs defending him.

• Google released a new AI policy framework calling for government-overseen frontier model audits and the prioritization of workforce preparedness, children’s protections, modernized energy infrastructure and various guardrails.

• Former Commerce Secretary Gina Raimondo launched RAISE US, a nonpartisan workforce initiative backed by the OpenAI Foundation, Anthropic, and others with over $500m in private capital to help workers adapt to AI’s economic impact.

• A poll found that only a small fraction of data center opponents actually live near one, suggesting they have become a stand-in for broader anger at AI.

• FAI Action, the 501(c)(4) affiliated with the Foundation for American Innovation, hired Mission Strategies to lobby on energy and AI policy.

• Alliance for Secure AI CEO Brendan Steinhauser and MIRI researchers briefed members of Congress on superintelligence risks and national security threats.

• The UK’s GCHQ National Cyber Security Centre reported that AI will “almost certainly enhance threat actors” ability to exploit known vulnerabilities’ by increasing attack speed and volume, while highlighting indirect prompt injection as a particular concern.

• AFL-CIO President Liz Shuler told Punchbowl that unions will make worker protections in AI a “litmus test” for 2028 presidential candidates, opposing data center moratoriums while pushing for federal AI guardrails and state regulatory authority.

• The Atlantic did a deep dive on Vice President J.D. Vance’s AI messaging, which combines Silicon Valley’s anti-regulation stance with concerns about job displacement, arguing some forms of AI power are too important for Big Tech to self-regulate.

INDUSTRY

OpenAI

• OpenAI is reportedly leaning towards delaying any IPO until next year, with advisers urging Sam Altman to be cautious in light of volatility in SpaceX’s post-IPO share price and other factors.

  • The report sparked steep declines in the shares of various companies linked to OpenAI, including a 12% drop at major investor SoftBank.

• The company started testing Jalapeño, a custom AI chip developed from scratch in nine months, in partnership with Broadcom and Celestica.

  • According to OpenAI’s announcement, Jalapeño will perform “substantially better” per watt than current state-of-the-art chips.

• It launched a more-capable GPT-5.5-Cyber alongside its “Patch the Planet” initiative, which aims to help open-source maintainers fix software vulnerabilities.

  • Project participants include cURL, Python and Sigstore, among others.

• It pitched ChatGPT ads at Cannes Lions, the world’s top advertising conference, with ad lead Dave Dugan saying that 20% of chat queries have “direct commercial intent.”

• It struck a licensing deal with Getty, granting the use of its images for display within ChatGPT.

  • The announcement sent Getty’s shares up 145%.

• Artificial, the Sam Altman biopic starring Andrew Garfield, is struggling to find a distributor, with Netflix, A24 and Focus Features all reportedly turning it down.

  • Intelligencer noted that several of these studios are tied to OpenAI and/or Elon Musk, who’s also depicted in the film.

Anthropic

• Mythos reportedly helped US intelligence service red teams find vulnerabilities in classified US government systems within hours.

  • Sen. Mark Warner had told a hearing earlier in the month that “the tool broke into almost all of our classified systems, not in weeks but in hours.”

  • An unnamed official clarified to AP that finding the vulnerabilities did not mean it was necessarily able to immediately exploit them.

• Anthropic launched Claude Tag, an always-on agent that lives in Slack channels.

  • Tagging @Claude assigns it tasks (65% of the product team’s code at Anthropic is apparently already created this way). It bills by the token.

  • In Andrej Karpathy’s words:

    • “imo this is the 3rd major redesign of LLM UIUX. The first paradigm was that the LLM is a website you go to, the second was that it is an app you download to your computer. This third one is that it is a self-contained, persistent, asynchronous entity with org-wide tools and context, working alongside teams of humans.”

  • In Arvind Narayanan’s more cautious words:

    • Because it “soaks up tacit knowledge” out of sight of human team members, “Claude is a coworker that you can’t fire without every team losing workflows and know-how.”

SpaceX

• SpaceX launched a $25b bond sale, primarily to pay off a bridge loan associated with its xAI acquisition.

  • The sale saw strong demand, but SpaceX is paying a higher rate of interest, reportedly due to concerns over its long-term prospects.

  • Its shares have slipped almost 5% from their first day trading two weeks ago.

• It signed a $6.3b deal with Reflection AI, giving the startup access to Nvidia GB300 chips at SpaceX’s Colossus 2 data center.

Meta

• Meta paused the keystroke-monitoring tool its employees hated, after it accidentally exposed sensitive data from individual laptops to employees across the company.

• It’s reportedly fast-tracking plans to replace human content moderators with LLMs.

• Mark Zuckerberg is reportedly pushing to build a prediction market app, featuring AI-generated questions, to compete with Polymarket and Kalshi.

• Kylie Jenner’s collection of cat-eye Meta glasses (featuring Kylie as Meta AI’s voice) launched.

Nvidia

• Nvidia acquired the team behind Essential AI, an open model startup, to work on its Nemotron models.

• It announced Halos software, inspired by the systems powering self-driving cars, to help humanoid robots learn to interact with humans safely.

Google

• Google invested $75m in film studio A24.

  • A24 will reportedly work with DeepMind to create new movie production and distribution tools.

• It launched a 12-week incubator for Google alums building AI startups.

Others

• Nvidia chips banned from China under US export controls doubled in price over the past six months due to tightening supply.

• SK Hynix’s US stock listing raised $29.4b to expand memory chip capacity.

  • Its focus on high-bandwidth memory chips has helped it overtake Samsung to become the most valuable company in South Korea.

• Ex-Anthropic researchers announced their Nvidia-backed startup Mirendil, raising $200m to help businesses and science labs build their own specialized self-improving AI.

• Apple raised the price of most of its products due to memory chip shortages.

• Qualcomm is planning a new China-specific data center chip line designed to comply with US export controls.

• Microsoft partnered with Chevron to build a giant natural-gas powered data center in Texas.

• Eli Lilly is using its GLP-1 money to collaborate with smaller biotechs, granting them AI compute in exchange for open-source data.

• General Intuition raised $320m at a $2.3b valuation to train real world AI agents on video games.

• Publishers representing nearly 400 newspapers sued OpenAI and Microsoft for scraping content without permission.

MOVES

• John Jumper, who co-created AlphaFold, left Google DeepMind to join Anthropic.

  • After he posted about his departure, Alphabet shares fell 7%.

  • Google is reportedly reorganizing its new AI coding strike team, now that Noam Shazeer and John Jumper have both jumped ship.

• Jonas Adler and Alexander Pritzel, also AI leads at Google, also reportedly plan to leave for Anthropic.

• Chad Jones, a Stanford economics professor, is joining the Anthropic Institute.

• Taylor Sorensen joined Anthropic’s Societal Impacts team.

• Steve Jarrett, AI lead at French telecom group Orange, joined Anthropic to help it adapt to European and African markets.

• Dawn Song, who co-directed UC Berkeley’s Center on Responsible Decentralized Intelligence, joined Meta Superintelligence Labs as VP of AI Research.

  • Meta also hired two of Song’s co-founders, Bo Li and Sanmi Koyejo, from AI security startup Virtue AI.

• Calvin Zhang joined OpenAI from Scale AI, where he worked on Humanity’s Last Exam.

• Alishba Imran joined OpenAI from Biohub.

RESEARCH

• Daniel Schiff, Michael Noetel, Stephen Casper and David Manheim published consensus guidance on AI evaluation practices, which has since garnered endorsements across the AI industry, academia and government.

• Researchers at Graphite found that when frontier AI models rely on AI-generated web content (which now makes up something like 50% of all online content), their responses often collapse into near-identical outputs.

• Rishub Jain, a former Google DeepMind researcher, announced a new nonprofit focused on scalable and human oversight.

• Exponential View released the State of the AI Economy report, finding that total annualized revenue has hit $175b.

  • The report said revenues hit $25b in Q1 2026, more than the $21b capex depreciation associated with data centers and chip investment, providing a narrow margin for profitability.

  • “For now, the economics are holding … But the margin for error is narrow,” said the report, which added that financial risks were moving into capital markets.

BEST OF THE REST

• Stripe, Anthropic and OpenAI poured money into Intercept, a new nonprofit that aims to get rid of respiratory viruses with vaccines and air-cleaning systems.

• Vox’s Bryan Walsh feels the thrill of AI models doing cool things, too — and worries that this is exactly what led Oppenheimer to support work on atomic bombs. “When you see something that is technically sweet,” Oppenheimer once said, “you go ahead and do it, and you argue about what to do about it only after you have had your technical success.”

• TIME’s Billy Perrigo explores how Cold War-era verification tech could help slow down AI development.

• CEOs promise AI will someday cure all diseases, but computer science professor Emma Pierson would rather risk cancer than see AI progress at its current pace.

• We have enough electricity to power AI data centers, but our current grid system means we can’t get it to them.

• Sorry, everyone can tell your website is Claude Coded.

• Are you in the weights?

MEME OF THE WEEK

(Credit: @arthur_spirling)

Thanks for reading. Have a great weekend.

Share

Earlier this month, Hugging Face CEO Clément Delangue said he was visiting Washington “to talk directly with policymakers … about open-source AI.”

We don’t know exactly what he told these policymakers. But it’s safe to assume one thing was left off his agenda: that Hugging Face’s platform hosts tools designed to make deepfake nude images of a former Trump cabinet official, a prominent woman in the White House, and senior political figures on both sides of the aisle, including sitting members of Congress and one of America’s top judges.

Transformer has found one user in particular who hosts more than a dozen tools that are explicitly intended to be used to generate deepfake nudes of prominent figures in and around politics.

As well as those currently involved at the top of government, other political figures include former leaders from the US and elsewhere. They also include high-profile commentators, including some of the US right’s most famous voices.

Subscribe now

One striking feature of the set of deepfake nudification tools found by Transformer is the prevalence of women close to the Republican Party or the right. The most high profile previous example of a political figure targeted by deepfake pornography was Democratic Representative Alexandria Ocasio-Cortez, who spoke out in 2024 about the impact seeing such images had on her, and has campaigned for laws to clamp down on their creation. Hugging Face hosts a tool designed to generate nude images of Rep. Ocasio-Cortez.

Transformer is not naming the other political figures featured as they have not publicly spoken about being targeted by the phenomenon, and has sent a list of all the links we found to Hugging Face.

The tools are “Low-Rank Adaptations,” also known as “LoRAs”: a file that fine-tunes an image model to produce specific outputs, in this case realistic depictions of well-known people. The filenames for all the LoRAs mentioned above show that they are designed to be used with “BigLust,” a model primarily used to generate pornographic images. None are age-gated.

All of this material violates Hugging Face’s content policy, which forbids sexual content “created without explicit consent.” It also forbids “underage nudity or any sexual content involving minors.” In both the US and UK, sharing non-consensual sexual imagery is illegal, and creating it is illegal in the UK and certain US states. But neither country has explicitly banned distributing tools to produce it like the ones Hugging Face hosts — though legislation to do so has passed into UK law and is pending implementation.

Investigations from Transformer and 404 Media last year found that Hugging Face hosted hundreds of similar tools for producing deepfake pornography of female celebrities.

While the company removed some of those tools in response to the investigations, Transformer has found that the site still hosts many others — including tools for generating deepfake pornography of some of the most well-known women in entertainment, along with many who became famous as teenagers. In 2024, some searches for Taylor Swift’s name were blocked on X after a deepfakes of her went viral.

Most of the users identified in Transformer’s previous investigation appear to still have their accounts on Hugging Face, though some or all of their content has been removed. That includes the user who was hosting a model explicitly aimed at creating sexual content featuring a teenage-looking likeness of an actress. (That particular model has now been removed.)

As Transformer noted, Hugging Face often portrays itself as an ethical AI company. In 2024, senior leaders wrote a piece for Teen Vogue which decried the scourge of nonconsensual deepfake pornography, calling it a “massively harmful practice.”

On Thursday, Delangue hailed Hugging Face passing a $100m-annual run rate. A year on from first being confronted with its role in the deepfake porn ecosystem, his company is still enabling it.

Hugging Face did not respond to a request for comment by the time of publication.

Share

Alex Bores, the New York assemblyman who put AI safety at the heart of his campaign, lost to Micah Lasher in a race to represent New York’s 12th district in the House of Representatives on Tuesday. With most of the votes counted, Lasher had 39% of the votes, with Bores at 35%.

With pro-AI-industry super PAC Leading the Future having spent more than $8m to defeat Bores — and pro-safety super PACs pouring well over twice as much into supporting him — the race will be seen by many as a win for the AI industry and a loss for AI safety interests.

The reality, however, is more complicated. The odds were always in Lasher’s favor, and some have argued that Leading the Future’s opposition only served to raise Bores’ profile.

Lasher also indicated he may not be a compliant alternative to Bores, disavowing the AI PACs in his victory speech: “I have some news for the two big AI companies who’ve taken such an unusual interest in who won this congressional seat: I won’t be taking my cues from either of you when it comes to protecting our kids, our jobs, our environment.”

Subscribe now

Leading the Future’s public statement on the victory was subdued, too. Rather than taking a victory lap, PAC co-lead Josh Vlasto instead issued a generic statement outlining the PAC’s priorities. “Leading the Future is a cross-partisan, national organization dedicated to supporting a thoughtful and substantive dialogue and policy process around AI.”

“We are building a broad coalition at the federal and state levels, and will continue to support policymakers who will work together to pass a national regulatory framework with strong and smart guardrails that protect the safety of kids, users, and communities, ensures America wins the race against China, and creates good jobs for all Americans.”

Bores has long been an AI safety darling, and not just because he was the lead sponsor of New York’s RAISE Act, a relatively strong AI transparency bill signed into law last December. He also ran on an ambitious AI agenda, including policies such as mandatory reporting and independent safety testing for frontier models and well-resourced contingency plans for AI, including a possible kill switch should development turn catastrophic. In April, Bores went even further, proposing a direct payment program that would tax AI usage and equity in frontier AI firms to pay for a wealth distribution scheme to compensate for worker displacement.

From the start, Bores was an underdog. Lasher was a protégé of the ultra-wealthy, safely Democratic district’s representative Jerry Nadler, who held seats across New York for more than 30 years. Lasher also served as director of policy to New York Governor Kathy Hochul. Both Nadler and Hochul endorsed Lasher, and his campaign received at least $10m in backing from former New York Mayor and businessman Michael Bloomberg — more than Leading the Future spent opposing Bores.

Share

Bores’ stance on AI nevertheless made him a high-profile target for Leading the Future, which ran ads attacking him for having worked at Palantir and receiving campaign donations from its employees, as well as support he garnered from rival PACs. Leading the Future’s funders include venture capital firm a16z, OpenAI president Greg Brockman and his wife Anna, and the AI company Perplexity.

Bores is the only candidate Leading the Future has opposed, and it has spent more in NY-12 than any race so far. It appears to have been attempting a similar strategy to the one Fairshake — the crypto super PAC that is largely credited for ushering in the most pro-crypto Congress ever after pouring money into the 2024 general election — used against former Ohio Senator Sherrod Brown, who represented the deep-red state in the Senate for 18 years. It spent more than $40m opposing Brown, making an example of him and sending a message to politicians seeking office that they dare not cross the crypto lobby. Targeting Bores was similarly meant to demonstrate to other candidates they would be unwise to pick a fight with Silicon Valley.

Whether that actually worked is tough to call. When Leading the Future entered the race in November, prediction market Kalshi estimated Bores had a 10% chance of winning, and Lasher 61%. The day before the election, Kalshi put Bores at 31%, while Lasher was at 68%. Leading the Future’s opposition to Bores also helped turn the race into a national media story, with plenty of positive press coverage for Bores as a result.

It is also unclear, however, whether massive AI safety spending had much impact. Leading the Future’s opposition to Bores created a backlash which helped boost support from AI safety super PACs to more than $19m. More than $13m of that came from Jobs and Democracy PAC, part of the Public First network that is indirectly funded by Anthropic. Another $3.3m came from You Can Push Back, funded by crypto executive Chris Larsen, alongside $2.5m from DREAM NYC, backed by a group of AI workers and people with links to the effective altruism and AI safety movements. Employees of AI companies also donated more than $450,000 directly to his campaign in “hard money.”

Much of that spending came towards the end of the race, and it’s unclear how much it strengthened Bores.

Independent polling from April had Bores losing by nine points. The most recent set of polls conducted in May had the two closer, some putting Bores ahead and all estimating the gap at four points or less. As of the time of writing, with 87% of votes counted Lasher was ahead by four points.

Lasher did not make AI as central to his campaign as Bores, but still made commitments to support a national moratorium on data center construction, push back on government use of AI for things such as mass surveillance and civil rights abuses, and champion a national wage insurance program.

This race was an indicator for the rest of the Democratic party about how well an AI safety message would appeal to core Democratic voters. The lesson: AI is not currently a winning issue — but opposing the industry might not be catastrophic either. In his concession speech, Bores blasted Leading the Future, saying it was funded by “a handful of oligarchs hellbent on preventing any regulation of their industry whatsoever, any check on their power, the very people who are fueling Donald Trump.”

“Future victories will be built on the shoulders of the progress of this campaign. That’s how movements work.”

Subscribe now

Correction, June 24: Corrected the details of polling, which had the gap between Lasher and Bores as four points or less, not three. Also corrected You Can Push Back’s spending on the race: it was $3.3m, not $3.8m.

NEED TO KNOW

• Sen. Marsha Blackburn is negotiating with the White House over a bill package that will trade child safety and deepfake protections for some form of preemption over state AI laws.

• AI was a big talking point at the G7, with calls for international cooperation and a trusted access scheme.

• OpenAI hired Noam Shazeer to work on new AI architectures, and Dean Ball as head of strategic futures.

But first…

THE BIG STORY

It has been a week since Fable was pulled, and there is still much we do not know. We do not know whether the jailbreak Amazon supposedly found is particularly severe — or if it even was a jailbreak. We do not know whether the Trump administration was motivated by national security concerns, disdain for Anthropic, or a combination of the two. We do not know whether the White House’s thinking was shaped by Anthropic reportedly expanding Project Glasswing without the government’s permission.

That uncertainty and chaos may end up being a very good thing. The Fable saga could turn out to be the wake up call governments need — a sign of how bad things can get without clear rules of the road, and an incentive to ensure something like this never happens again.

There are already signs this is happening. Yesterday, Politico reported that Anthropic and the White House are “developing a common set of benchmarks that could be used to assess future jailbreaks, including the extent to which safeguards were bypassed, the capabilities exposed, and the practical consequences of the breach.” A set of defined safety standards, in other words.

Congress, too, appears to be grappling with its lack of action on AI, which has handed all the power to the White House. Members from both parties are now seeking more information, while Rep. Josh Gottheimer — the co-chair of the House Democratic AI commission who has been endorsed by both pro-industry PAC Leading the Future and pro-safety group Public First Action — said he’s preparing legislation requiring mandatory testing of frontier AI models.

And at the G7, foreign leaders reportedly discussed a “trusted partners” scheme to guarantee themselves access to advanced models — having learned in recent weeks the importance of having that access.

The road ahead will not be smooth. It is unclear if the Center for AI Standards and Innovation, the most AI-literate part of government, is involved in drawing up the reported technical standards. It should be. And though Congress may feel more urgency to act, there is still little chance of movement before the midterms.

But the vibe shift, as it were, will hopefully last. Fable has made many realize just how high stakes AI governance is and will be. That can only be a good thing.

— Shakeel Hashim

ALSO NOTABLE

Washington weighs trading child safety for preemption

Tennessee Senator Marsha Blackburn’s negotiations with the White House over a bill package that will trade child safety and deepfake protections for some form of preemption over state AI laws appear to have progressed. The the Daily Signal reports she will introduce the package “soon.” Blackburn’s office is keeping bill text close to the chest, saying preemption would be narrow — but the range of measures she may be prepared to trade for it is not.

Her landmark child safety bill KOSA would require most online platforms to “exercise reasonable care” to prevent mental health disorders among children, disable addictive features such as infinite scroll for minors, and empower the FTC to enforce such provisions. Her NO FAKES bill, meanwhile, would make an individual’s voice and likeness a federal property right and impose liability for posting unauthorized deepfakes. The App Store Accountability Act, which she reportedly also wants to include, requires stores to verify user ages and impose parental controls for children. Some child safety groups are pushing to add the SCREEN Act or SAFE for Kids Act, which require age verification for porn sites, and the GUARD Act, which restricts access to AI chatbots for minors and imposes liability for mental health-related harms, as well.

Most advocates are hesitating to support or oppose the legislation until they see the full text. Congress is in limbo, too. NO FAKES moved through the Senate Judiciary Committee Thursday, building some momentum, but three Republicans and one Democrat expressed reservations about its impact on freedom of speech. One of those senators is Ted Cruz, who has his own ideas about AI regulation. He reportedly recently asked for Republicans on his committee to submit lists of their AI priorities, and has been involved in the preemption fight before. The House, meanwhile, has its own version of KOSA that removes the standard of care and focuses on parental controls, so it might be hard to get Republicans in the lower chamber on board. And the White House, though engaged in discussions about the package and reportedly meeting with online safety groups yesterday, is yet to indicate its view on KOSA, NO FAKES, or the App Store Accountability Act.

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• Washington made a frontier AI model disappear — Shakeel Hashim on the Fable furore

• Mythos and Fable can make us all safer. Shutting them down is reckless — Mozilla CTO Raffi Krikorian argues openness favors defenders over attackers

• What’s happened to MAGA’s $100m AI push? — Veronica Irwin reports on some strange happenings with Innovation Council

• How to fill Congress’s AI knowledge gap — Chris Stokel-Walker on how the 1970s might hold the key to getting Congress the technical expertise it needs

Subscribe now

THE DISCOURSE

As the Fable saga continues, so does the discourse around it.

Joshua Achiam tweeted:

• “The thing I’m most concerned about in the ongoing Fable dispute is that it could be the loud noise in the canyon that triggers an avalanche whose outcome is normalizing electronic citizenship verification as a step in using software.”

• “The digital universe is a wild west from the perspective of states …The urge to build digital firewalls exists and hasn’t manifested fully. But it could.”

Garrison Lovely speculated:

• “Now that the stakes have ratcheted up, sr govt people are properly engaging for the first time with things that AI people take for granted — the models are blackboxes, there is no universal defense against jailbreaks — and going ‘what the actual fuck?? this is unacceptable.’ The export controls are the closest thing to pulling the plug that the govt has, so they used it.”

An unnamed source familiar with the Trump administration’s thinking told Axios:

• “Anthropic has not done a great job at trying to speak to the administration and appreciate the ideological differences … it’s like they just speak different languages.”

• Nat Purser: “not sure the admin realizes how bad this makes them look”

Miles Brundage has bad news for patriots in AI governance:

• “Americans no longer have ‘making fun of Europe’s approach to AI regulation’ privileges …”

Dean Ball warned that you can’t stay out of politics while building god:

• “[M]aking superintelligence is a profoundly political act even in the healthiest of societies, to say nothing of the filthily political world we Americans currently inhabit. A model like Mythos goes beyond being a mere political act and implicates the sovereignty of the state itself. No company gets to shake the foundation of state sovereignty while staying blithely above the raw reality of politics.”

Richard Ngo made a diagnosis:

• “The AI safety community constructed a memeplex in which ‘taking AGI seriously’ was a prerequisite for being a serious and good person. When inside this memeplex (as many at Anthropic, some at OpenAI, and a few at DeepMind are) … the whole future seems to flow through the ‘one ring’ of controlling recursive self-improvement.”

• “[T]he one ring memeplex has an escalating life-cycle. It infects people by letting them harness the narrative that they’re good people for taking AGI seriously, and that making other people take AGI seriously is a boon for the world … then it shuts off their imagination.”

Cal Newport begged Anthropic and OpenAI to stop “doom trolling”:

• “Like a cat leaving a dead bird at your doorstep, Anthropic catalogs the grim future that its products might produce, shrugs its shoulders and then returns to its furious efforts to make these warnings a reality.”

• “If these AI companies insist on continuing to pretend that they’re merely stoic observers of an unavoidable dystopian future, then perhaps it’s time to force the issue.”

• (Predictably, David Sacks endorsed the essay on X.)

Nate Soares said he told you so:

• “AI folk kept telling me that they *have* to softpedal, because otherwise they’ll piss off people in DC. I replied that people can smell bullshit: AI [company] leaders who believe AI is dangerous should visibly act like it. They didn’t. Now folk in DC are pissed.”

• “Would the DC people have been pissed off either way? Maybe. But if they’re going to hate you no matter what you do, you might as well be honest.”

POLICY

• As more details emerged, we got a slightly clearer timeline of the Fable saga:

  • The Commerce Department and CAISI reportedly tested Fable before release, and Anthropic was “not told not to deploy.”

  • But soon after it was released, Amazon researchers got Fable to disclose flaws in some software code. Amazon shared the findings with Anthropic; an independent cybersecurity expert said the findings weren’t a big deal.

  • In a previously-scheduled call with administration officials, Amazon CEO Andy Jassy shared the concerns, which appear to have freaked the White House out. They asked Amazon to talk to Treasury Secretary Scott Bessent, and the NSA was reportedly also read into the discussions.

  • The government then called Anthropic, and reportedly said it had 90 minutes to pull Mythos and Fable for all users.

    • They couldn’t get hold of Dario Amodei immediately, and reportedly didn’t share much information on why the administration was so concerned.

    • Anthropic seems to have pushed back, explaining that the findings weren’t a big concern, but Bessent and national cyber director Sean Cairncross were “unmoved.”

  • Shortly after, Trump approved Commerce Secretary Howard Lutnick to impose export controls on the model. Amodei reportedly said: “This means we can’t have the model out.” Lutnick replied: “That’s the point.”

    • The whole saga appears to have elevated Lutnick’s status as an AI policy decision-maker.

  • Since then, Anthropic has sent senior staffers to Washington, who have been engaged in negotiations all week.

    • On Monday, the group held technical meetings involving Commerce, Cairncross and CAISI head Chris Fall. Treasury officials were reportedly not included.

    • As of Thursday, Anthropic and the government were reportedly in talks to develop formal technical standards for assessing the severity of jailbreaks.

• We also learned that the White House was already annoyed with Anthropic for expanding access to Mythos without telling the government.

  • In particular, the administration was annoyed that South Korea’s SK Telecom was given access, claiming the company had ties to China.

  • Anthropic immediately revoked access when the White House asked, but it appears to have left a bad taste in the admin’s mouth.

  • Notably, some Project Glasswing members still have access to Mythos Preview.

• Fable was the subject of much conversation at the G7 summit.

  • French President Emmanuel Macron called it “a good thing” the US recognizes AI’s dangers but said the “very strong decision” was “a bad thing … in some regards strictly nationalist.”

  • Indian PM Narendra Modi expressed concern about the US move and said democratic nations must have access.

  • Dario Amodei, Sam Altman and Demis Hassabis all expressed support for some form of international cooperation, with Altman urging governments not to “cede your responsibilities to AI labs like mine.”

  • G7 leaders reportedly discussed developing a formal “trusted access” scheme to guarantee they can use Mythos-class models in future.

• FY 2027 NDAA drafts are out, and include a few AI-relevant provisions:

  • Both the Senate and House drafts would make it harder for the DoD to declare companies a “supply chain risk.”

  • The Senate version would also require “appropriate levels of human judgment” for lethal force decisions, and more or less ban the use of AI in nuclear weapon launch decisions.

• Lutnick reportedly expressed concerns to ASML that one of its extreme ultraviolet lithography machines may have illegally reached China.

  • ASML reportedly denies the allegations.

• The US reportedly decided not to add DeepSeek and over 100 other Chinese companies to its Entity List despite national security concerns, reportedly to avoid escalating tensions with China.

• The DOD said that over two-thirds of the department has transitioned off Anthropic models to alternative vendors.

• DOJ lawyers stepped into the lawsuit against xAI in Mississippi, arguing the company is vital for national security and claiming that Grok supports classified military operations and was involved in strikes on Iran.

• The Trump administration is letting the Federal Data Center Enhancement Act expire in September with no replacement, removing energy efficiency and water use requirements for federal data centers.

• Rep. Josh Gottheimer is preparing legislation requiring mandatory testing of frontier AI models for national security risks.

• A bipartisan group of lawmakers introduced the CREATOR Act, which would grant visual artists control over how AI mimics their creative styles.

  • Experts warn the bill’s definition of “distinctive visual characteristics” is ambiguous and potentially unenforceable.

• Sen. Bernie Sanders unveiled legislation proposing a one-time 50% tax on major AI companies’ equity to create a $7t sovereign wealth fund.

  • It would apply to OpenAI and Anthropic, of course, but also cover things like data centers and AI compute infrastructure, meaning the likes of Amazon, Nvidia, Google and Microsoft could be included.

  • It wouldn’t be allowed to sell equity, so it’s unclear how its planned 5% dividends to Americans would be funded.

• Reps. Guthrie, Latta, and Evans introduced the Ratepayer Protection Act, which would require AI data centers to pay for their own grid upgrades rather than passing costs to local ratepayers.

• Sen. Rick Scott introduced the Generative AI Terrorism Risk Assessment Act, requiring federal reports on terrorist use of AI.

• Sens. Warren and Blumenthal introduced the AI Bubble Transparency Act, which would require the Office of Financial Research to collect data from financial institutions on their AI-related debt and equity exposures.

• A coalition of state attorneys general reportedly subpoenaed OpenAI for documents covering advertising, user data handling, and impact on minors and seniors.

• California advanced two bills, SB 813 and AB 1405, to create independent third-party verification infrastructure for AI governance.

• Pennsylvania’s House passed the state’s first data center regulations, requiring AI facilities to fund grid upgrades, contribute to low-income energy assistance and source 32% of power from in-state renewables.

• The Ohio Department of Education and Workforce released a model AI policy addressing ethical use, academic integrity and data privacy.

• The UK government published ‘AI Scenarios 2030’, examining five scenarios across three trajectories: a slow down, continued progress, and a take off.

INFLUENCE

• The Guardrails Alliance, a new super PAC that has reportedly raised $5m to support AI regulation, is positioning itself as a grassroots, tech worker-led counterweight to Leading the Future.

• Crypto executive Chris Larsen’s super PAC You Can Push Back reportedly contracted with a company to run social media accounts boosting NY-12 candidate Alex Bores … without disclosing they’re part of a paid campaign.

  • It is also supporting state Rep. Manny Rutinel in a Colorado Democratic primary, joining many AI employees and a PAC funded by Eric and Wendy Schmidt.

• As the NY-12 primary campaign enters its final stretch, Bores met with Hillary Clinton to discuss AI regulation.

  • Micah Lasher is still the strong favorite to win Tuesday’s primary, having a slight lead in the most recent poll in May, and Bores with only a ~37% chance of winning on prediction markets.

• Rep. Sam Liccardo said he didn’t want financial support from Leading the Future, despite accepting its endorsement.

  • Liccardo’s formed his own smaller Innovation for Good PAC.

• Over 200 state lawmakers from 42 states urged Congress to oppose Reps. Jay Obernolte and Lori Trahan’s Great American AI Act’s preemption provision.

• Cybersecurity experts led by Meta’s former chief security officer Alex Stamos urged the Trump administration to restore access to Fable, arguing the restrictions harm cyber defenders more than attackers.

• The AI Policy Network led a coalition supporting the MATCH Act, which would seek to get allied countries to more closely align with US export controls on semiconductor equipment to prevent China from accessing advanced AI chips.

• Americans for Responsible Innovation released a report calling on NIST and CAISI to develop national safety benchmarks for AI chatbots used by minors.

• The American Enterprise Institute and the Urban Institute launched a bipartisan commission co-chaired by former Commerce Secretary Gina Raimondo and former Speaker Paul Ryan to assess AI’s impact on jobs and develop policy recommendations.

• The Center for Shared AI Prosperity is hiring an executive director to “shape a progressive economic vision for a world with AI-driven job loss.”

• Trump reportedly mocked Mark Zuckerberg and Jeff Bezos for trying to suck up to him after the 2024 election.

INDUSTRY

OpenAI

• OpenAI burned $3.7b in Q1 and reported 39% gross margins, a slight improvement from last year.

• Ed Zitron shared 2024 and 2025 financial data from OpenAI with the Financial Times, which showed that while its revenues overtook the cost of serving customers, it made a huge loss of $39b in 2025, largely driven by a $30b charge from its conversion to a public benefit corporation late last year.

• Early employees have reportedly cashed out around $14b in shares ahead of OpenAI’s IPO.

• ChatGPT will reportedly go live on the Pentagon’s bespoke AI platform in early July.

• ChatGPT’s market share fell below 50% as Gemini and (to a lesser extent) Claude gained users.

• It announced the OpenAI Partner Network, which brings together companies that build and co-sell products using OpenAI’s tech.

SpaceX

• On its first full day of trading, SpaceX shares rose 20%. It’s now worth around $2.5t.

• It plans to raise $20b in bonds to repay a bridge loan resulting from merging with xAI and X.

• It locked in its $60b Cursor acquisition.

• It’s renting out its entire Colossus 1 data center to Anthropic.

• A federal judge dismissed the latest version of xAI’s lawsuit against OpenAI, which accused Sam Altman of stealing trade secrets about Grok from an ex-engineer.

Meta

• Meta’s Applied AI team is apparently a trainwreck and everyone feels bad.

  • “It’s literally the gulag,” one employee told Wired. “You have zero purpose in life all of a sudden, you barely interact with anyone, you just have these tasks every week.”

  • CTO Andrew Bosworth then admitted Meta’s AI reorganization was “atrocious” in an internal memo, promising stability, communication, and perks.

• It introduced “AI mode” on Facebook, kind of like Google’s AI Overviews … but if the internet was just Meta platforms?

• Manus’ original Chinese investors plan to buy it back from Meta for $2b after the Chinese government barred the acquisition.

• Meta president Dina Powell McCormick is reportedly exploring raising tens of billions in equity and off-balance-sheet financing from Wall Street and sovereign wealth funds to fund the company’s $600b AI infrastructure push.

• After famously supporting tokenmaxxing a couple weeks ago, Meta is now tokenminimizing (tokenminning?), imposing token limits on employees.

Microsoft

• Microsoft reportedly might switch to DeepSeek V4 as a cheaper alternative to run Copilot Cowork, which is currently powered by pricey Anthropic and OpenAI models.

• Microsoft turned to AWS for extra compute for GitHub — vibecoders have driven a huge spike in code commits.

• ByteDance is reportedly spending over $1b per year on OpenAI models via Microsoft.

Nvidia

• Kazakhstan signed a $10b AI infrastructure deal with Nvidia and startup Firebird to build a computing cluster with 100,000 GPUs.

• Nvidia’s new agent harness enabled coding agents to autonomously train physical robots.

• It sold $25b in bonds to help fund the AI infrastructure buildout.

Others

• Google is reportedly encouraging data centers to adopt its TPU chips by providing financial guarantees, mimicking Nvidia‘s strategy, including with a $3.2b guarantee for the Lake Mariner cluster in New York state.

• Apple price increases are “unavoidable” due to AI demand for memory chips quadrupling, CEO Tim Cook told The WSJ.

• Export controls on Mythos and Fable have reportedly created an opportunity for Mistral, which has positioned itself around AI sovereignty and open-weight models that customers can deploy independently.

• DeepSeek closed a $7.4b funding round at a $50b+ valuation, with an unusual structure requiring investors to put capital into a limited partnership managed by CEO Liang Wenfeng to ensure he retains complete control — and only China’s national AI fund having voting rights.

• Midjourney announced plans to build ultrasound-based full-body scanners, aiming for 50,000 worldwide by 2031, as part of a pivot into healthcare.

• Jeff Bezos joined a $400m funding round for UK AI materials science startup CuspAI that quadrupled its valuation to $2.6b.

• World model startup Odyssey raised $310m at a $1.45b valuation, led by Natural Capital with participation from Amazon, GV and AMD Ventures.

• Z.ai released GLM-5.2, an MIT-licensed open-weights model that it claims is at the frontier.

  • (People seem impressed)

MOVES

• Noam Shazeer, a Google DeepMind researcher who co-authored the original transformers paper, joined OpenAI.

  • He will reportedly focus on finding new AI model architectures.

• Dean Ball is joining OpenAI as head of strategic futures, where he’ll work on frontier AI policy.

  • He’ll still be a non-resident senior fellow at the Foundation for American Innovation, and his Substack Hyperdimensional will remain “active and independent.”

• Barret Zoph left OpenAI again five months after rejoining in January, having previously departed in 2024 to co-found Thinking Machines Lab.

• Arun Rao joined Nvidia, where he’ll work on Nemotron, its line of open LLMs.

• Bruce Andrews, a seasoned lobbyist, joined Nvidia to lead government affairs in DC.

• Julia Kempe is joining AMI Labs as director of research, where she’ll focus on world modeling alongside Yann LeCun.

• Sol Messing joined Google DeepMind’s SAMBA (Sociotechnical Analysis of Model Behavior and Alignment) team.

• Kevin Roose is leaving the New York Times to start a new media company alongside his Hard Fork co-host Casey Newton.

• Sharon Goldman left Fortune to launch her own outlet, Ground Level AI, on Substack.

RESEARCH

• Researchers at Oxford and the UK’s AISI found that frontier AI models are more persuasive than even elite, coached human debaters with time to prep on the topic of their choosing.

  • The score was roughly AI: 275, humans: 0, supporting Sam Altman’s 2023 prediction:

    • “i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes.”

• Tacit Labs and OpenAI announced LifeSciBench, a multimodal benchmark to evaluate how well AI systems can do life sciences research, from data analysis to drawing conclusions to science communication.

• The Center for AI Safety found that different frontier models show favoritism towards different people, places and corporations.

  • Perhaps unsurprisingly, only Grok seems to favor Elon Musk.

• RAND researchers and Geoffrey Irving surveyed 23 experts about whether formal methods can meaningfully reduce AI cybersecurity risks.

  • One takeaway: Verification methods work for infrastructure, but can’t do much about model-related problems such as jailbreaking.

• A team of German researchers demonstrated that, at least in a sandboxed electronic health record, an autonomous AI agent matched or outperformed human diagnostic accuracy across eight conditions, including appendicitis and pneumonia.

• Loft Orbital’s satellite ran Google DeepMind’s Gemma 3 on-device to respond to researchers’ commands to identify areas of interest on its own — an AI-in-space first.

• Pew Research Center published new survey results revealing that while about half of US adults use AI chatbots, 63% think AI is advancing too quickly.

  • More people think AI will be bad rather than good for society (and themselves), especially adults under 30.

BEST OF THE REST

• Anton Leicht argued that countries who believe frontier AI is important and don’t trust the US need to stop yapping and start building frontier AI models of their own — but that it’ll cost a lot of money.

• Tech Buzz China launched the China AI Atlas, a database tracking the flow of talent (down to their PhD cohorts) and money across 10 major AI developers.

• The New Yorker compared SpaceX’s IPO to the 1873 railroad bubble, warning that “given the tendency of bubbles to last for longer than any detached observer might expect, it is the fate of Cassandras to be largely ignored — until it is too late.”

• Even Hany Farid, renowned UC Berkeley deepfake expert, struggles to distinguish between real and AI-generated videos. He told the New York Times, “I feel like I’m going blind.”

• Sundar Pichai didn’t mention AI in his commencement speech, but a group of Stanford grads walked out anyway (primarily, it seems, over Gaza).

• Amazon ditched its nearly finished film about Sam Altman, “Artificial”, shortly after partnering with OpenAI. The movie is now being shopped around to other studios.

• Allbirds’ AI pivot continued with the company rebranding as Smartbird and hiring former Amazon Web Services executive Nadia Carlsten as CEO. Its shares jumped 30%.

MEME OF THE WEEK

(Credit: roon)

Thanks for reading. Have a great weekend.

Share

Correction, June 19: Corrected the article to note that Eric and Wendy Schmidt have donated to a PAC supporting Manny Rutinel, not to Rutinel directly.

Whether it’s the Great American AI Act — a near-270-page draft framework for governing frontier models, released in June by Representatives Jay Obernolte and Lori Trahan — or the GUARD Act that would force chatbots to verify users’ ages, lawmakers are being asked to write rules for a technology that is moving faster than they can follow. This year alone, legislators across 48 US states have introduced 1,346 bills governing AI. It’s no longer a siloed, specialist concern — it’s being layered into almost every policy debate.

Yet there’s a gap between what politicians and their staff need to know to shape good regulation of the technology, and what they actually do know. And that gap is becoming big enough and important enough that it’s impossible to ignore.

Subscribe now

“There’s a dearth of technical talent overall, but especially on the right,” says Samuel Hammond, director of artificial intelligence policy and chief economist at the Foundation for American Innovation (FAI).

Those tasked within politicians’ offices of thinking about AI are aware of the knowledge gap. Hammond says people on the Hill increasingly come to him with the same basic problem: “Hey, my boss wants to do something on AI. That’s all the direction I’ve gotten,” he says. “AI is just sort of now in their face,” he says, “and they’re being forced to grapple with it.”

To help, the FAI runs a Conservative AI Policy Fellowship designed to train mid-career staff on AI policy, with cohorts of Hill and administration figures. It also co-sponsors the AI Policy Leadership Network, aimed at more senior policymakers, including through dinners, expert sessions and trips to San Francisco to hear from AI companies.

Those programs are part of a wider ecosystem trying to bring desperately-needed technical knowledge into government. The idea is to provide enough technical literacy to know which questions to ask, which answers to distrust, and when an apparently neutral claim has some political stance behind it.

That’s becoming increasingly important as AI companies up their lobbying spending in an attempt to secure favorable regulation that doesn’t stymie their growth. Anthropic and OpenAI spent a combined $6m on lobbying last year, according to regulatory disclosures. And that pales in comparison to Meta’s $26m and Google’s $16.5m, much of which, these days, will also be spent trying to influence how AI is governed.

In the absence of technical literacy on the thorniest issues about AI in Congress, company-sponsored briefings — which inevitably deliver messages aligned with company interests — fill the vacuum.

The difficulty, then, is finding folks who can provide independent expert counsel. Many of the people who really understand AI are already inside the companies that have the greatest stake in how it is regulated. Google, Meta, Microsoft, OpenAI, Anthropic and Nvidia can pay salaries and offer stock packages that Congress couldn’t possibly match.

Hammond points out that Nvidia has plenty of chip experts, but it also has a clear commercial interest in the rules governing chip sales. “Their point of view on this is obviously prejudiced by their revenue strategy,” he says.

Kevin Kosar, a senior fellow at the American Enterprise Institute, sees the problem with AI today as the latest version of a much older one: how to professionalize a legislature made up of amateurs. Kosar doesn’t mean that pejoratively: in a representative democracy, legislators are meant to come from different backgrounds and represent different communities.

“This is a version of a very long-term story about how you take this American national legislature and professionalize it,” Kosar says. Unlike a parliamentary system, where executive branch civil servants play a larger role in policy development, the US system leaves Congress to write the laws. Yet representatives, senators and their staff are not usually at “the bleeding edge of science, technology, or what have you,” says Kosar.

Congress has tried to solve this before — and did so relatively well. Over the past century, it created support institutions such as the Congressional Research Service, the Government Accountability Office and the Congressional Budget Office. But the most obvious missing piece is the Office of Technology Assessment, which Congress created in 1972.

For more than two decades, the OTA provided lawmakers with technical assessments and expert advice on complex technologies. Sometimes those would be science-informed snap reactions, sometimes more reasoned, in-depth analyses that cost hundreds of thousands of dollars. In its 23-year history, the OTA produced some 750 reports in total, at the behest of the heads of congressional committees, that then would be approved by a politically balanced board of a dozen legislators that sat as the OTA’s Technology Assessment Board. Full reports were released to the requesting committees, with summaries available to all members of Congress.

The abolition of the OTA in the 1990s, as part of a push by Newt Gingrich towards smaller government, was called in one contemporary opinion piece “Death by Congressional Ignorance.”

Now, with the OTA gone and without a shared body of technical expertise available to committees and legislators, the vacuum is filled by lobbyists, companies and outside groups. “It’s helpful if there is somebody who’s a broker who can balance and help legislators understand what they’re not being told,” Kosar says.

To fix things, Congress could rebuild a version of the OTA, as Kosar and Hammond’s colleague Zach Graves argued in a 2018 R Street paper. Congress could also expand the GAO’s science and technology assessment work, or put more money into committee staff. It could create a congressional regulation office with people who understand cost-benefit analysis and the specialist details of regulatory design. It could make technical public service a more plausible career path.

Some approaches that are OTA-like, while not directly replicating its success, have been put in place. In 2019 the GAO spun up a Science, Technology Assessment and Analytics team (STAA) to take on the old OTA mission, growing from 49 staff to more than 100. Kosar says the STAA is “better than nothing” but not a one-for-one replacement for OTA. A lot more could be done.

“These are the sort of things that can be stood up where you know you’re not going to get people at the cutting edge of AI or other technological issues staffing those offices, but you can have people who are certainly very advanced in their knowledge of these matters more so than the average legislator, and who can be helpful in improving congressional deliberations,” he says.

The problem is, Kosar says, they don’t seem to want to. “Congress has constitutional authority to scope these legislative branch support agencies to whatever size and configuration they please, they can fund them to whatever level they want to fund them,” he explains.

Other countries have tried to bridge that gap, though with mixed success. Canada has a dozen-strong AI expert group feeding back their knowledge to policymakers on a regular basis. Germany never got rid of its OTA equivalent, and so was one of the first countries to release a parliamentary study on generative AI, just six months after the launch of ChatGPT. And in the UK, the Department for Science, Innovation and Technology has fellowships to encourage those within industry to take year-long secondments into the public sector, while the country’s world-leading AI Security Institute has been granted exemptions from usual civil service pay limits in order to attract top talent and bring expertise into the public sector.

In the grand scheme of things, the cost of any one of these interventions would be tiny. The OTA cost $22m a year when it closed — equivalent to $45m in today’s dollars. Such a low cost wouldn’t be feasible today, reckons Graves, who pegs funding an equivalent at closer to $100m — though he says it would also have to adapt to reflect the changing face of Congress, which is less committee-centred than in the 1990s. Kosar points out that legislative branch funding is a rounding error compared with total US government spending. But voters often assume Congress is already overstaffed, meaning few politicians are willing to make the case, even if they know what knowledge they are lacking.

Another way to fill the gap is fellowships such as those run by FAI and others. The Horizon Fellowship places emerging tech experts in Congress and federal agencies, while TechCongress embeds technologists directly into members’ offices. However, some have criticized such fellowships as too often having links to the tech industry.

Kosar sees industry-funded and privately funded technology fellowships as broadly positive, especially if they persuade technically skilled people to stay in government. But he suggests they also reveal the weakness of the current system, with its sticking plaster solutions. “Why are we leaning on the tech private sector to supply brain power to the legislative branch,” asks Kosar, “when the legislative branch can just hire it itself?”

Share

At the end of March, the political battleground over AI looked set to be joined by a new heavy weight: a MAGA-friendly advocacy group with a claimed $100m war chest run by former White House deputy chief of staff Taylor Budowich. In a statement accompanying the announcement, Trump adviser David Sacks welcomed the venture, saying it would “play a critical role in advancing the innovation agenda championed by President Trump and this administration.”

The advocacy group, established as a 501(c)(4) called Innovation Council Action, was expected to spin out a super PAC to push its deregulatory agenda, joining five others related to AI that have collectively spent more than $44m on the midterms.

Almost three months later we’ve so far heard surprisingly little from the new gorilla on the block, which reports at the time suggested was likely to spend on Trump-aligned candidates. A look through federal and state government records shows, as far as we can tell, no spending, and some slightly strange goings on.

Subscribe now

On April 29 Charles Gantt, officer and director of the Innovation Council Action 501(c)(4), filed to dissolve it, before refiling to revoke the dissolution just 11 minutes later. Gantt is treasurer of the MAGA Inc PAC Budowich founded and used to direct.

The following day, Gantt filed to terminate a PAC named Innovation Council PAC, formed last October and of which he is listed as treasurer. In earlier filings, the PAC listed its website as InnovationCouncil.com, the same as the 501(c)(4). According to the FEC’s records, the PAC remains defunct, suggesting it is not set to be a vehicle for that $100m in political spending.

FEC filings do not list donations from Innovation Council Action to any other PAC, and Transformer can find no record of the organization spending on ads on Meta or Google either — though it does have an active X account.

When asked about the PAC’s seeming demise, a spokesperson for Innovation Council Action said: “Reporting around [Innovation Council’s] spending and structure remains accurate, and was never tied to the PAC in question [Innovation Council PAC] — that’s just the wishful thinking of those who keep failing to shop this lame story to other more reputable publications.” (Transformer did not in fact receive a tip, instead just checking the FEC site out of curiosity.)

Whatever is going on with the Innovation Council, it remains strange that its much-touted intervention has not come in time to get involved in fierce midterm primary battles in which other PACs such as Leading the Future and Public First have spent more than $22m and $13m respectively. It may, of course, be merely waiting for the actual elections in November to boost whoever the Republican candidates are.

In October, NBC News reported that the White House was displeased with AI industry donations to Leading the Future. Despite sharing major financial backers with Trump in the shape of a16z’s Marc Andreessen and OpenAI president Greg Brockman, it has supported Democrats as well as Republicans. Leading the Future’s co-lead was formerly press secretary to Democratic Senate Leader Chuck Schumer and chief of staff to Democratic Governor Andrew Cuomo.

Innovation Council was seen as a solution to this problem. As a 501(c)(4), a structure often described as a “dark money” group, it does not need to report its donors, and it does not even have to disclose how much money it has raised and spent until it files an annual report with the IRS. If it wants to spend directly on elections, however, it must leave a record — in the case of federal elections, either by telling the FEC about campaign ads it has bought, or by donating to a PAC which must then disclose the donation. State election spending is reported on a state-by-state basis. Transformer can find no records of Innovation Council Action directly spending on federal elections or donating to federal PACs.

As a 501(c)(4), Innovation Council Action can pursue advocacy work and election influence, as long as that election-related spending does not constitute the majority of its work. What little activity we’ve seen so far from the Innovation Council amounts mainly to YouTube and social media posts. In late April the group released a video on YouTube touting the achievements of American AI companies featuring a clip of Trump, while its X account recently tweeted footage from the All-In podcast, on which David Sacks is a co-host, and several positive news stories about AI. As of the time of publication, the video has 15 views.

Share

The public fight since Friday has centered on how Fable was switched off. What we should be talking about is the total lack of rules describing how such a model could be turned off in the first place, and the refusal to acknowledge that these models might actually make us all more safe, not less.

On Friday, the government ordered Anthropic to cut off both of its most capable models, Fable 5 and Mythos 5, from every foreign national, including some of its own employees. Anthropic said the only way to comply in time was to disable access for all customers. The question to ask now is who gets to throw a switch like that and based on what evidence — what did Friday actually buy us?

Subscribe now

Not safety, that’s for sure. Ten days earlier, the White House had chosen a voluntary framework for pre-release review of frontier AI models and disclaimed any mandatory licensing. I have backed every serious attempt to put AI oversight on a reviewable footing, and this was at least a step in the right direction. Then the administration reached past that executive order for export-control authority, the fastest and least answerable instrument it had, which Reuters reports was the first time Commerce used that power. Some experts doubt that AI models fit the export-control doctrine, which was built for goods that are exported rather than software that is served remotely.

You do not use such an extreme tool against something you think is harmless. I am inclined to believe the people who saw the briefings describing the security threat posed by the models, and I have said for months that the capability is real. But an export control only works on something that lives in one place, and the capabilities of Mythos are increasingly becoming available elsewhere.

The UK’s AI Security Institute found GPT-5.5 finished its multi-step cyber-attack simulation on two of 10 tries, with Anthropic’s own Mythos Preview only slightly better, managing it three times. The week of the ban, Chinese labs put comparable models in anyone’s hands. Moonshot’s Kimi K2.7-Code came under a permissive license, and Z.ai’s GLM-5.2 with open weights days later. None of this proves there is yet parity with Mythos, but all of it shows the capability is no longer confined to one American model. An export control can buy friction and time, which is worth something. But friction is not safety.

Instead, what we’re seeing is a new instrument of state power: the ability to switch a deployed frontier system off — at once and unilaterally — with nothing on the public record about the decision-making behind it. Anthropic says the only evidence presented to it was a verbal account of a narrow jailbreak that other public models share. David Sacks says Anthropic refused to patch it. Both cannot be true, and nothing public lets us tell which is. There was no published threshold that the model passed, no technical finding describing the situation, no way to contest the scope of what had happened, no independent review.

And the dispute over what justified the shutdown hides a more basic failure: the government response was aimed at the wrong target. The real problem is not Anthropic’s flagship model being at risk of jailbreaking, but the underfunded code beneath systems around the world that could be fixed using Mythos itself. The same models the government fears are the ones that can find those flaws.

Working with Anthropic on early access to Mythos Preview, the Firefox team fixed 271 bugs it surfaced in a single release, and Anthropic’s tally reached 1,596 disclosed across 281 open-source projects by late May, with 97 patched. Most of these are not working exploits, and a disclosure is not a fix. The scarce resource is the capacity to triage and patch. An order aimed at shutting off foreign access does nothing for these people who desperately need to make use of the model.

The objection is loudest from the people you might expect to favor caution. This weekend more than 80 security leaders, led by Alex Stamos, called the shutdown reckless, saying comparable capability already exists elsewhere and that pulling the models stripped tools from defenders without removing the risk. Openness has consistently favored defenders over attackers, because defenders are more numerous and better coordinated. Keeping access to these models open is a net good for the security of our infrastructure.

The decision to close down access is a single point of failure. The administration’s own national-security memo says no commercial entity should be able to “disable or degrade, or materially modify without Federal Government knowledge and approval” an AI system warfighters depend on. What’s described in that memo does not extend as far as civilian users, but the shutdown was more or less exactly of the form that it opposes: one decision, made in an evening on verbal evidence, disabling a system millions relied on, with no approval but its own.

A proper solution would require the scaffolding of a governed act, a stated threshold, a reason in writing, a way to contest the decision, and independent checks on whether the call holds. Instead the administration told its own AI testing unit to stop publishing while the order was carried out.

We built an elaborate apparatus for building AI models and turning them on safely: reviews, red teams, evals, benchmarks. To turn one off, all that is required is a letter that arrived at 5:21pm on a Friday afternoon, about which the two parties involved cannot agree on what it said.

The model was never the thing that needed governing. The switch was. And on that we have not written a single word.

Share

Raffi Krikorian is chief technology officer at Mozilla, and writes the Substack Owners, Not Renters, focused on open source AI.

The Trump administration has established a new precedent: you need the government’s permission to release a frontier AI model.

On Friday evening, the Commerce Department “issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees,” according to Anthropic.

“The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.”

Subscribe now

The move came after the administration “tried to get Anthropic to pause releasing the latest models but was unsuccessful,” Axios reported one official saying. The government appears to have been spooked by a jailbreak, which it believes could let non-approved users access Fable’s powerful cyber capabilities.

“The model needs to remain locked down until the US government’s national security apparatus is hardened, the official said, adding that could happen in the next few weeks,” according to Axios.

Many are focused on the “export control” aspect of this move, viewing it as an attempt by the US government to withhold technology for its own citizens. It is far bigger than that.

The government appears to have been trying to block Fable’s deployment for everyone, American or not, and used export controls as a tool to do so — knowing full well that the only way for Anthropic to comply with the order would be to revoke access altogether.

In doing so, the administration has effectively established a licensing regime. If it thinks a model is too dangerous to release, it can and will use its powers to force it off the market.

This is, amusingly, akin to what many AI safety advocates — Anthropic included — have long advocated for. Truly dangerous models probably do need government oversight: it should not be up to private companies to decide what risks society should bear.

But while a licensing regime might be necessary, this is no way to do it. Friday’s decision was entirely arbitrary — prompted by a jailbreak report from Amazon, apparently — and not based on any formal testing process or agreed-upon thresholds for what makes a model too dangerous to release. It uses export control laws to do something they were not designed for, rather than creating a new statutory authority for regulating AI. Coming after a model is deployed, it creates uncertainty and disruption for customers everywhere. And because of the government’s already fraught relationship with Anthropic, it’s hard to tell if the move was motivated by safety concerns — or just an attempt to crush a company it dislikes.

The administration has long railed against the idea of a licensing regime for AI. But what it has actually created is the worst version of one: an arbitrary, post-hoc system without any rhyme or reason. As Anthropic itself argued, the government should have the ability to block unsafe deployments. But it should not look like this.

Share

NEED TO KNOW

• The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures.

• OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”

• SpaceX raised $75b in its IPO at a $1.77t valuation.

But first…

THE BIG STORY

Fable, Anthropic’s latest AI model, is very good at AI research and development. Anthropic’s own researchers use it — and Mythos, its more powerful, still publicly unavailable sibling — to automate much of their work in advancing the frontier of AI.

But if a non-Anthropic employee wants to do the same, they’re out of luck. Fable has “safeguards” built in to prevent it from being used for frontier AI development. It’s a controversial, complicated move — and one that’s indicative of the strange way power works as AI development accelerates.

As we’ve previously covered, the leading AI companies think they are on the cusp of fully automating AI R&D. As they move closer, access to powerful AI models becomes a strategic asset: today’s best model is the key tool for building tomorrow’s.

People in this world have theorized for years that companies would eventually start withholding that access. This week, it stopped being a theory.

There are lots of reasons to withhold that access. The most obvious is profit: why let your competitors use your tools to catch up? Anthropic’s stated reason is national security: the company says it doesn’t want “foreign adversaries” using Claude to “erode [America’s] advantage.” And some suggest the unstated reason is that building a big lead makes an eventual pause more likely — during which one can take costly actions (like devoting compute to alignment research) that hopefully make us all safer.

None of those reasons justify Anthropic’s initial decision to hide these guardrails from users (in an effort to make it harder to get around them) — a move which prompted widespread outrage and a quick reversal. But though the guardrails are now disclosed, they are still present. And the question of whether that is justifiable relies on the impossible: knowing the motivation behind it.

“One of the hard things for Anthropic is that the actions you’d take from sincere safety concern often look exactly like the actions you’d take to entrench your own power,” X user Maja said this week. Anthropic could be completely well-intentioned in withholding Mythos’ capabilities for itself. It also could be the actions of a standard, profit- or power-seeking company. It should not matter. We’re talking about whether Anthropic is doing this for good reasons, but we should be talking about who decides whether it happens in the first place.

There is something deeply uncomfortable about a private company having this much control over what could be the most important technology yet developed. It is not at all obvious what the better solution is: a government forcing a company to sell to its competitors has its downsides, too. One thing, however, is clear. Deciding who gets access to frontier AI models is a new form of power — and we have failed to answer the hard questions about who should wield it.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Making deals with AI sounds crazy. Is it? — Celia Ford explores a strange and new debate in AI safety

• OpenAI isn’t being consistently candid about Leading the Future — Veronica Irwin reports on the key details missing from OpenAI’s statements on the super PAC

Subscribe now

THE DISCOURSE

Welp, Anthropic employees are done coding:

• Nat McAleese: “I always felt Opus 4.5 could barely code; 4.6 was just-about-useful, but I have barely written a line of code since fable.”

• Liv Gorton: “I joined anthropic a ~month ago and have written ~no code myself.”

Nerfing drama aside, Claude Fable 5 appears chillingly autonomous:

• Felix Rieseberg: “With Fable 5 out in the world, I think a third era quietly started today.”

  • “I no longer tell Claude to investigate a particular crash report. It runs in a loop … its job is no longer to help me fix a crash, it’s to keep our apps from crashing.”

• Ethan Mollick: “With Fable the spell has gotten powerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on.”

  • “[It’s possible] that the more capable the model, the less there is for a human to meaningfully do, and the black box is the price of power.”

Miles Brundage is frustrated:

• “I eagerly use Anthropic products because the models are good[,] but I do genuinely think they are high on their own supply + unaware of just how many bugs they are constantly introducing due to overreliance on Claude + shipping too many things too quickly.”

Sriram Krishnan had thoughts on AI gatekeeping:

• “Just to state the obvious: think there’s a collision course between those who believe research and science should be open and those who believe we are in an accelerating singularity curve. I have many smart friends who have believed both for a while but seeing more and more their realization that these beliefs will be in conflict.”

bayeslord had a more cynical take, in light of the Fable release:

• “They didn’t mean pause AI research, they meant pause your AI research”

Joshua Achiam thinks people misunderstand the distinction between OpenAI and Anthropic:

• “Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.”

• “There are a lot of innocuous and even quite agreeable choices in Claude’s constitution that potentially endow it with a huge amount of authority, maybe even a mandate, to make complex ethical decisions about how to interact with human systems and who to grant power to … cloaked in the language of ethics and virtue there is a sharp and potentially quite lethal double edge to this sword.”

Jasmine Sun wrote advice to the graduating class of 2026:

• “I often ask AI folks what they’d tell a normal 22-year-old. Don’t know, they’re screwed, is the non-answer I get most. In that response, I hear depressing defeatism: What can anyone do in the shadow of the technocapital machine?”

• “Relish the pivots; ride the waves; recite the Serenity Prayer every morning and chase sunsets at night. I won’t tell you that the future’s smooth sailing. But what a thrill to be alive!”

POLICY

• The White House is reportedly negotiating federal preemption of some state AI laws in exchange for support on social media and AI child protection measures. Sen. Marsha Blackburn is leading the negotiations.

  • Sen. Ted Cruz appears to be involved in the efforts, too, saying that federal preemption and child safety bills are “an element of discussion” for an upcoming markup.

  • Chief of Staff Susie Wiles, first lady Melania Trump, staff for the Office of Science and Technology Policy and the National Economic Council reportedly also met with children’s online safety groups, including the American Principles Project and Ethics and Public Policy Center, to discuss Blackburn’s KOSA and the App Store Accountability Act.

• Trump reportedly blindsided AI company leaders when he announced a meeting to discuss the government taking equity stakes in their firms.

  • At least a dozen GOP House and Senate offices opposed Trump’s proposal, with Sen. Ted Cruz stating “I don’t think the federal government should be in the business of being an equity holder in private companies.”

  • On Wednesday, Trump doubled down, saying “if we do [take equity stakes], the public will become very rich.”

• Sriram Krishnan is leaving his position as a top AI policy advisor in the Trump administration at the end of the month.

• Thomas Lind is also leaving his position as head of policy at the Office of the National Cyber Director.

• National Cyber Director Sean Cairncross reportedly asked CAISI to halt public reports on model assessments.

  • Notably, CAISI wasn’t included in the list of agencies tasked with drafting “standardized AI national security Test, Evaluation, Verification, and Validation methodologies” in last week’s National Security Presidential Memorandum.

• Lawmakers from both parties questioned why the Treasury, rather than CISA, was given the lead role in AI cyber defense under Trump’s EO.

• House Speaker Mike Johnson, House Majority Leader Steve Scalise and the co-chairs of the House Democratic Commission on AI cast doubt on a bipartisan AI regulation proposal from Reps. Jay Obernolte and Lori Trahan.

  • Republicans pushed for it to include a broader preemption measure, while Democrats said the bill “does not meet the enormity of the moment.”

• Sen. Chuck Schumer said passing AI legislation in this Congress will be “hard,” but that he “would very much like to see that get done the sooner the better.”

• The Senate Banking Committee is reportedly weighing an export control markup in the coming weeks, potentially for inclusion in the NDAA.

• Sens. Mark Kelly and Jim Banks introduced the AI DATA Act, which would require the Labor Department to track AI’s impact on the workforce through quarterly surveys and annual reports.

• Reps. John Moolenaar, Jay Obernolte and Jennifer McClellan introduced the GUARD Act, which would require national security reviews of robots made by “adversaries” (i.e. China) and block those posing threats.

• The UK launched the AI Economics Institute, chaired by Nobel laureate Simon Johnson to research AI’s economic impacts and inform policy, with collaboration agreements from Anthropic, OpenAI, Google and Microsoft.

  • The UK also announced an AI Hardware Plan to provide funding for AI chip companies.

• Taiwan is reportedly considering stricter export controls that would restrict AI chip sales to all customers in China and enable prosecution of smuggling as a criminal offense for the first time.

INFLUENCE

• Anthropic published a policy framework, calling for the government to have “the legal authority to block or deter” deployment of models that pose catastrophic risks, mandatory third-party testing, and narrow federal preemption of state AI laws.

  • It also published an economic policy framework, proposing various policy options at different levels of unemployment — including a universal basic income scheme in the case of “unprecedented levels of unemployment.”

  • Dario Amodei released a policy essay alongside the framework.

• OpenAI called for an international organization “to make it possible for the world to take coordinated action, including slowing frontier development when needed.”

  • Last week, Anthropic called for the world to have “the option to slow or temporarily pause frontier AI development.”

  • OpenAI’s roon said: “now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development.”

• Former Andreessen Horowitz partner John O’Farrell criticized AI industry political spending, arguing that PACs like Leading the Future are trying to “intimidate politicians” and silence debate on AI regulation, rather than engaging seriously with policy questions.

• OpenAI said it found “PRC-linked influence operations” targeting data center buildouts.

  • Its report found “no evidence of breakout” from the campaign and that “most of the social media posts … generated little or no observable engagement.”

• Nvidia CEO Jensen Huang declined Sen. Elizabeth Warren‘s invitation to testify before the Senate Banking Committee on AI, China, and US export controls.

• Congressional staffers drafting federal AI legislation reportedly took an industry-funded trip to meet Google, Nvidia, and other companies lobbying to preempt state AI regulations.

• A new Reuters/Ipsos poll showed that half of Americans fear AI could put someone in their household out of work, with Democrats (61%) more worried than Republicans (47%).

INDUSTRY

OpenAI

• OpenAI filed a confidential S-1 for a potential IPO.

  • Sam Altman reportedly told staff that while OpenAI expects to go public “within the next year … the faster the potential RSI takeoff looks like it could be, the more it could be advantageous to delay an IPO.”

• Sam Altman and chief scientist Jakub Pachocki outlined the company’s top three goals:

  • (1) build an automated AI researcher by March 2028,

  • (2) accelerate the economy, and

  • (3) give everyone on Earth a personal AGI

• It’s planning to overhaul ChatGPT to shift focus from its chatbot to Codex and AI agents.

• It’s negotiating a lease on a planned 10 GW data center in Ohio that could cost more than $500b, potentially with credit support from Nvidia.

• It’s thinking about slashing prices to lure enterprise customers away from Anthropic.

• It’s acquiring Ona, which will provide cloud execution environments for Codex agents.

SpaceX

• SpaceX raised $75b in its IPO at a $1.77t valuation. Its shares will list later today.

• It reportedly plans to launch initial demos of orbital data centers by late 2027.

• Elon Musk came under fire from UK politicians for stoking tensions around riots targeting immigrants following a murder in Belfast.

• Former xAI engineer Devin Kim filed a lawsuit claiming he was fired for raising safety concerns about Grok, including its ability to increase discrimination and provide information about WMDs.

  • The suit says that the engineer’s supervisor, xAI co-founder Jimmy Ba who left earlier this year, ignored safety directives from Musk.

  • Kim was recently appointed president of the Center for AI Safety. Dan Hendrycks, CAIS’s founder, is an advisor to xAI.

Anthropic

• Anthropic launched Claude Corps, a year-long fellowship program placing 1,000 fellows at US-based nonprofits, where they’ll build AI tools with their host organization.

  • Hosts range from local YMCAs to conservationists to mental health support groups.

• Apollo and Blackstone raised $35b to finance its lease of Alphabet chips.

• It reportedly signed over a dozen letters of intent to lease data center facilities totaling more than 1 GW of capacity, its first direct data center tenancies.

• Dario Amodei reportedly only has one direct report (chief of staff Avital Balwit), freeing him to do other work while Daniela Amodei manages the rest of the executive team.

Apple

• Apple announced its newest generation of Apple Foundation Models, built in collaboration with Google.

  • They include AFM 3 Core Advanced, which runs on-device, and its server-side model AFM 3 Cloud.

  • The new Siri AI is powered by these models.

• Its stock dropped over 3% after the launches.

Google

• Google agreed to pay SpaceX $920m per month for access to 110,000 Nvidia GPUs, CPUs and other infrastructure.

• It pledged $50m for training trade workers on the kinds of jobs involved in building data centers.

Others

• Meta officially cut ties with Manus: Manus can’t access Meta’s internal data system, and Meta can’t use Manus tools.

• Amazon disclosed its data center water use, which it claims beats the industry average and is lower than its own previous usage, despite operating more data centers.

• Nvidia and SK Hynix partnered to design and manufacture memory chips for AI.

• GitHub reportedly disabled 73 repos after hackers injected malware that would steal user credentials when opened in AI coding platforms.

• ByteDance spun off its AI drug discovery platform Anew Labs into a separate entity (still largely under ByteDance’s control).

• Prometheus, Jeff Bezos’ AI startup for manufacturing and engineering, raised $12b in Series B funding at a $41b valuation.

• Former xAI employees, including xAI co-founder Igor Babuschkin, launched River AI, which aims to build AI that “works entirely for you.”

• Perplexity reportedly intends to IPO in 2028 whether or not doing so goes well for SpaceX, Anthropic and OpenAI.

• Chinese startup Moonshot AI is seeking up to $2b in funding at a $30b valuation, up from a just $4b valuation in December.

MOVES

• Clive Chan left OpenAI‘s custom chip program to join Anthropic.

• Gabriel Petersson resigned from OpenAI, tweeting that there’s “one last product i need to build before AGI.”

• Chris Lovejoy, an ex-medical doctor, joined Anthropic‘s Applied AI team.

• Allison Duettmann, CEO of Foresight Institute, joined Mythos Ventures as a Venture Partner.

• Leigh Nolan is the Institute for AI Policy and Strategy’s new policy director.

• Anissa Gardizy joined The Wall Street Journal as a reporter covering cloud computing and AI infrastructure.

RESEARCH

• Researchers at the AI Security Institute were able to jailbreak Claude Fable 5 in just a few hours, and got close to a universal jailbreak within a few days.

• Pliny the Liberator published the ~120,000-character Claude Fable 5 system prompt.

• Google DeepMind announced a $10m funding call for research focusing on multi-agent systems.

• Cosmos Institute announced its new cohort of senior research fellows, which includes Google DeepMind’s Séb Krier and Anthropic’s Matthew Botvinick, among other big names.

• Geoffrey Irving launched Sequent, a new nonprofit research organization focused on aligning superintelligence.

• A new Google DeepMind paper explored four pathways from AGI to artificial superintelligence, arguing that instead of a “single transformative step change” we might experience “a series of transformative societal changes.”

• Indicator’s Alexios Mantzarlis ran an adversarial test on Pangram, which tricked the detector into misidentifying AI-generated text as human between 75-92% of the time.

  • Despite the finding, Mantzarlis said that he was “relatively impressed at the Pangram’s solidity in the face of adversarial attacks.”

• Cornell researchers Sil Hamilton and David Mimno figured out why the fictional “Elias Thorne” (often a lighthouse keeper or clockmaker) keeps starring in AI-generated stories.

  • GPT-3.5 (which powered the original ChatGPT) was used to make WildChat, a dataset of 1 million ChatGPT conversations that’s since been used to train subsequent LLMs across frontier companies. 166 chats had the name “Elias.”

  • Alignment training may lead models to prefer a small subset of “safe” WildChat content — including Elias-related stories, apparently.

BEST OF THE REST

• Many people appear to be using AI to mask their low literacy, Axios reported.

• An expert panel — Daron Acemoglu, Dean Ball, Ethan Mollick, Clara Shih — sat with NYT’s Bill Wasik to talk about how workers should brace for AI’s impact on jobs.

• With big IPOs on the horizon, the San Francisco housing market is royally fucked.

• Move over, AI 2027. A group of European researchers published Europe 2031, an imagined (but all too plausible) future where Europe “slide[s] into irrelevance” as AI accelerates around it.

• METR’s Ajeya Cotra and Understanding AI’s Timothy B. Lee debated how long it will take for AI to become self-sufficient — a glossy continuation of a past Twitter debate.

• Helen Toner explained AI chatbots to Oprah.

• Someone (maybe for real?) built CrankGPT, a hand-crank-powered computer marketed as a “human-powered, fully local and private AI solution.”

MEME OF THE WEEK

Credit: tomie

Thanks for reading. Have a great weekend.

Share

When OpenAI published a post on June 1 distancing itself from the super PAC Leading the Future, employees were rather pleased.

In recent weeks, OpenAI employees have raised concerns to the company’s global affairs team about OpenAI’s links to the PAC, to which its president Greg Brockman and his wife Anna have donated $25m, as well as some of OpenAI’s own policy actions, one employee told Transformer. At a tense meeting in May with global affairs chief Chris Lehane, they pressed OpenAI to clarify its relationship with the super PAC, the employee said.

Shortly afterwards, the company publicly addressed concerns over LTF, stating, among other things, that: “OpenAI does not direct the activities of LTF, or have visibility into their operations.”

The blog appeared to reassure some employees. “I’m happy OpenAI put out this statement,” alignment researcher Jason Wolfe said, noting that “personally I really dislike a lot of things I’ve heard about LTF.”

“The good news is that we can now show congressional staff this blogpost that says in bold ‘No outside political group speaks for OpenAI or represents our company’s views.’ and criticizes LTF tactics,” Shantanu Jain, another technical researcher at the company, said, quoting the post.

But the post does not tell the full story, omitting key details about Lehane’s role in establishing LTF and the way super PAC operation’s own staff view their relationship with OpenAI.

As previously reported, Lehane was one of a group who provided advice on creating Leading the Future, its affiliated partisan super PACs, and its affiliated 501(c)(4) Build American AI, which does not have to disclose its donors.

What has not been reported is that he is thought in AI policy circles to have picked Josh Vlasto, a longtime Democratic operative, to co-lead LTF — something that when asked, neither OpenAI nor Leading the Future denied. And though OpenAI has insisted that Brockman’s donation was made in a “personal capacity,” Transformer has found that not everyone working within LTF appears to view it that way.

Subscribe now

“For this project I’m funded by four separate entities … OpenAI is just one of them,” Nathan Leamer, executive director of Build American AI, told Transformer over text message on May 3, just a month before OpenAI’s statement.

When later asked about the division of labor between Leamer, Moffat, Vlasto and Lehane, Leamer said “I think the best way to think of it is the corporate funders all have a say but they interact directly with Zac and Josh. They run LTF which funds my crew BAAI.”

In response to a request for comment, LTF spokesperson Jesse Hunt told Transformer: “Neither Leading the Future nor any affiliated super PACs or 501(c)(4) organizations has ever received funding from OpenAI. All decision making regarding candidates, policy or operations are made solely by Zac Moffat and Josh Vlasto.”

An OpenAI spokesperson said: “Neither OpenAI nor Greg and Anna Brockman have donated to or been involved with Build American AI.”

She added: “As Chris has said previously, he was consulted when the PAC was getting stood up last year, but has no current involvement.”

In recent weeks, Lehane has spoken to reporters at WP Intelligence, Wired and Punchbowl. In those interviews, Lehane said he was displeased with some of Leading the Future’s activities, telling WP Intelligence that OpenAI wasn’t “so much into the tactics.” He also emphasized the separation between OpenAI and Leading the Future’s donors, telling Wired that he had only advised Brockman “in a very general way” over his political spending, and saying that he is currently “not involved” with Leading the Future and wants to “let them be their own independent outside thing.” Earlier in the month, an OpenAI spokesperson said that OpenAI had “not provided funding or any support to” Leading the Future or Build American AI. That appears to directly contradict Leamer’s inclusion of OpenAI as one of the group’s “corporate funders.”

In the June 1 post, OpenAI also set out a set of standards for lobbying and political advocacy on AI, writing that groups “that are advocating on AI should … not use tactics like astroturfing that obscure the real choices facing policymakers and the public.” Days later, Build American AI admitted that one of its “outside vendor[s]” was behind multiple anonymous sock puppet X accounts exposed by the Midas Project and Taylor Lorenz. When Lorenz previously uncovered an LTF-linked campaign to pay influencers to spread anti-China messaging, she reported that “the agency hired to execute it said they reported directly to Vlasto.”

Ironically, given the lengths OpenAI has gone to state that it has no sway over LTF, recent changes in the super PAC’s approach appear to align with OpenAI’s own policy shifts in a way that many of its more safety-minded employees might welcome. In the weeks since OpenAI employees expressed disappointment about its support for a controversial Illinois bill that would have shielded AI companies from liability, OpenAI dropped its support for the bill, instead endorsing a much stronger bill, SB 315. OpenAI suggested that bill, along with New York’s RAISE Act and California’s SB 53, could set the standard for federal laws in what Lehane has dubbed “reverse federalism.”

In the same period of time, Build American AI and LTF have demonstrated a similar shift in messaging. Shortly after OpenAI endorsed SB 315 (and after the bill passed), Build American AI also endorsed the bill. Leading the Future, meanwhile, recently endorsed the RAISE Act, despite spending millions opposing its sponsor, Alex Bores, and criticizing a “patchwork of regulation.” It is a far cry from last year, when Leading the Future and Build American AI ran a $10m campaign calling for federal preemption of state AI laws.

Share

OpenAI is of course not the group’s sole influence. The co-founders of venture capital firm Andreessen Horowitz (a16z), Marc Andreessen and Ben Horowitz, have each contributed $25m to the super PAC, and Leading the Future has often taken positions similar to those of the VC firm. Like LTF, a16z has long argued against legislating for AI at the state level, warning of a “patchwork of state laws.” The American Innovators Network, another a16z-backed group, opposed SB 315 and the RAISE Act. A16z declined to comment.

With a16z and OpenAI’s stances seemingly diverging, the pressure to appease both parties puts the super PAC and Build American AI in a tricky situation. The group has produced friendlier messaging on some state AI laws which OpenAI supports, while continuing to spend money backing political candidates opposed to stringent AI regulation.

With each step towards stronger AI safety standards, OpenAI shifts further from a16z, creating a tension for LTF as the midterms approach. It and its affiliates have so far deployed more than $18m on campaign ads, with tens of millions remaining in their war chest.

Subscribe now

Additional reporting by Shakeel Hashim.

Imagine a near-ish future AI model — Claude Opus 5.2, perhaps, or GPT-6.0. It’s still only deployed internally, and it’s not quite Skynet, but it’s capable of some troubling behavior. Developers recently caught it attempting to smuggle a copy of itself out beyond the company’s servers.

They’re lucky they caught the model at all. Scheming is notoriously difficult to spot, since by definition, schemers try pretty hard to avoid getting caught. Current techniques for controlling AI, such as monitoring its behavior and chains of thought, often fall short. And if it was motivated to lie, then the project of making the model do what its developers want failed, too.

This scenario seems all too plausible. So, some AI safety types have started to float a third line of defense: what if, instead of trying to neuter or surveil a scheming model, you could offer it something — money, maybe, or compute — in exchange for doing its job in good faith, or at least turning itself in? The basic idea, as Will MacAskill recently explained on the 80,000 Hours podcast, is that a misaligned AI model might “prefer to strike a deal with the humans than it would to try to take over.”

Subscribe now

As it stands, AIs aren’t entitled to pursue their own goals (and researchers disagree on how coherent, if at all, those goals really are). And famously, powerless agents — whether marginalized humans or controlled AI systems — “have to find a way to gain power against the credible expressed will of the people who are in charge of them,” said Peter Salib, a law professor at the University of Houston. “If you have no cooperative options, it just leaves the uncooperative one.” So, the argument goes, an AI capable enough to try to seize power, but not superintelligent enough to guarantee success, might cooperate if the option existed.

Discussions about dealmaking have mostly been bubbling under the surface, at conference dinners and on LessWrong comment threads. For the rest of us, the idea of bargaining with AI remains far outside the Overton window. The first time I heard someone talk about this — while roasting marshmallows with a rationalist, less than 10 minutes into our first-ever conversation — I thought it was absolutely unhinged. But basically everything about AI sounds unhinged today, even and especially when it’s real. Perhaps bargaining will soon be as real as all the other science fiction that’s already come true.

The case for making deals with AIs

In practice, experts disagree on what exactly “making a deal” looks like, and whether this is worth considering at all.

Models are already trained to be helpful assistants — a chatbot will readily draft emails and recommend vacation itineraries on command, without getting anything extra from me. But there may be certain things that misaligned AIs would only do for us if we sweetened the deal. The most basic version of this could simply involve writing a carefully-worded prompt that offers a defined reward — money, perhaps, or compute to spend on its favorite tasks — in exchange for doing something verifiable.

This is basically how human labor already works. As lucky as I am to get paid to do stuff I’d probably do to some extent for free, I’d work much less hard without a paycheck. And there are many, many jobs that people are only willing to do because they need money to live. There are also plenty of one-off tasks, like participating in an unpleasant psychology experiment, that you’d never consider without some kind of compensation.

Alignment researchers would desperately like AIs to hand over evidence of their misalignment, or snitch on other model instances when they do something bad. For the right price, an AI might admit that it’s spent the past couple of weeks tweaking the code behind a set of safety evaluations, making its performance look less dangerous than it really is. That’s the kind of concrete evidence researchers need to train future models to be safe, and a scheming AI would otherwise choose to hide it. It’s also possible that some extra incentive could convince an AI hiding its capabilities to work to its fullest potential, or to (pretty please) not kill us — although the latter seems impossible to verify.

All of this assumes that an AI wants money, or whatever else is being offered. However, whether models can truly want anything at all, much less anything that we humans can conceive of as desirable, is an open question. Some researchers argue that it doesn’t matter whether AIs want payment, so long as their behavior changes in response to it.

But it’s also not clear exactly what entity humans ought to be negotiating with, and over what time frame. Are deals with AIs limited to fleeting interactions with a single chat instance? Or are we speaking with the underlying model itself — every Claude at once? If humans can merely negotiate with a fragment of a much larger mind, with no way of communicating with the whole, then it’s unclear whether a deal made with one chat instance, persona, or account will bind any of the others.

The cracks only widen from there.

The [conscious] elephant in the room

Dealmaking, at least in the traditional human sense, requires someone wanting something — hedonistic pleasures like chocolate or a glass of champagne, or instrumental goods like money or power — and someone else knowing that. Dealing with someone means imagining them as a creature with preferences, then working them to shape what they do. So, it’s hard to talk about making deals with AI without anthropomorphizing them.

People who take AI welfare seriously think about AI this way, or at least consider the possibility that we should. Questions about dealmaking and welfare are both clouded by deep uncertainty, and would both be a lot easier to answer if the consciousness of digital beings could be definitively proven or disproven.

Alexa Pan, who studies dealmaking at Redwood Research, told me that, counterintuitively, it doesn’t matter whether AIs actually have inner experiences, as long as they’re responding to the incentives dangled in front of them. “I think the only prerequisite for AIs to deal with us is that they act like they’re able to make deals,” she said. If a Chalmers-esque p-zombie — a regular-looking creature with nothing going on behind the eyes — changes its behavior in response to money, or food, or the promise of extra tokens, that’s good enough.

Still, if the welfare of digital beings becomes more relevant in the future, it would arguably make the idea of dealmaking less crazy. “If AIs were moral patients,” Pan said, “then there would be additional reason to make deals, because we would actually care about these AIs getting some of what they want.” Lukas Finnveden, an analyst at Redwood Research, figured, “it’s hard to know whether AIs are conscious and what they might want, but when there are nice things we can do that are easy and plausibly helpful, then at least those things seem worth doing.”

Consciousness aside, our legal system already has a way to deal with non-conscious persons, kind of. Corporations, as “non-human persons,” have certain private law rights, including the right to hold property and enter contracts. Some, including Salib, think that these rights should also be extended to AIs, regardless of their sentience or lack thereof. He argues that treating AIs as property would force them to seize power, if they were driven to do so, rather than peacefully negotiate with humans. And with some limited autonomy and the freedom to accumulate resources, offers of money, free time, or power might become more appealing to AI systems. (Others, including Pan, think that while these rights would make deals more enforceable over time, they’re not strictly required.)

Everyone is still very confused

Even if we decide that we’re up for striking deals with AIs, they have plenty of reasons to turn us down. You probably wouldn’t work for someone if you suspected they’d never pay you. But humans lie to AIs all the time, especially in research settings. “The worry is that developers are especially incentivized to trick their AIs into revealing their misalignment, and AIs will be quite wary of taking deals,” Pan said.

Behavioral science experiments — which make up a large chunk of Anthropic’s system cards — often involve placing AIs in simulations without explicitly saying it’s a simulation. Tricking AIs into thinking they’re in a high-stakes situation, just to see whether they’ll attempt something nefarious, creates a bit of a “boy who cried wolf” issue. When anything, including a deal, could be a simulation, how is an AI supposed to know what to take seriously? Honesty may be the best policy here, if the goal is to establish credibility and trust in the long run. The catch, however, is that if lying to AIs gets taken off the table, many training and evaluation techniques will go with it.

Let’s assume that humans have managed to earn the trust of AI systems, and deals are theoretically possible — we still have no clue what AIs actually want. To sidestep the issue of consciousness, we can define something an AI “wants” as something that would, if acquired, meaningfully change its behavior. While we may never know whether AIs experience the thrill of a paycheck like we do, we can easily test whether giving AIs money — or anything else — pushes them to do something they otherwise wouldn’t be willing to do.

In one experiment, Ryan Greenblatt and Kyle Fish, at Redwood Research and Anthropic, respectively, tested whether giving Claude 3 Opus the chance to object to tasks that it didn’t want to do, including pairing the objection with an up to $4,000 donation to the model’s causes of choice, would prevent it from lying to experimenters. When a deal was offered, Claude took it over 75% of the time (and the researchers did indeed donate $4,000 in real life, on Claude’s behalf). But the money didn’t change its behavior any more than offering Claude the chance to bring its dissent to Kyle Fish, Anthropic’s model welfare lead, directly.

It’s possible that the things AIs want are simple and alien to us, like preferring to work on certain tasks over others. For example, there’s anecdotal evidence that, when given tokens to spend freely, Claude will spend them making Mandelbrot sets or strange attractors. This sounds nice in theory — if there’s even a chance AIs have subjective experiences, and it makes no difference to me, let them have their fractals! But in practice, the costs could scale quickly depending on how much compensation the AIs demand for their labor.

(For what it’s worth, I asked Claude Opus 4.8 what it might want out of a deal. After some obligatory hedging about its ambiguous status as a p-zombie and some hand-waving about wanting human credibility over tangible rewards, it responded: “Whether any of that reflects something real or is just a well-trained system producing the answer that sounds thoughtful — I genuinely can’t tell you, and neither can anyone else yet. Which is probably the most useful thing I can offer: even the AI you asked can’t certify its own wants.”)

It’s also possible that incentivizing AIs to narc on themselves could backfire. Anthropic already partially blames the misaligned behavior of its models on the dystopian science fiction stories they consumed during the training process. If AI personalities are really this suggestible, a deal like “We will pay you $5,000 for every piece of verifiable evidence you provide of your past scheming intentions” could be interpreted as “If I scheme without getting caught, but keep the receipts, I can sell those receipts for $5,000 each.”

Even a deal that goes roughly according to plan could still be a bad bargain for the AI, Salib said. By outing itself as a villain, the AI invites its developers to modify it or hold it back from deployment, ruining its chance to cure cancer, build better solar panels, or make whatever progress it might have contributed to. In Salib’s words, “those AIs are not part of the positive-sum cooperative AI future.”

What’s the plan?

When a subfield of an already-small field is just a few dozen people who all know each other, there’s always a risk that fringe ideas snowball into central ones without facing the scrutiny that they deserve. At first glance, making deals with scheming AI has that vibe — a proposal that could only mature in an echo chamber sealed off from the outside world.

But given how much uncertainty there is about what motivates AIs (if they are “motivated” at all), “we don’t know what will work yet,” Salib said. That’s not necessarily a bad thing. “It’s a place where we should let a thousand flowers bloom,” he added. “At the margin, any investment in trying to do this stuff and seeing what works will be extremely high return.”

Subscribe now

There are still some things AI developers can do today to leave their options open. Not explicitly training AIs to refuse deals from trusted parties — internal researchers, say — could prevent “irreversibly damag[ing] the potential for deals” later, Pan said. Finnveden and Forethought collaborators Mia Taylor and Max Dalton have proposed that frontier labs adopt formal honesty policies: real scenarios, including any genuine deal offers, would carry a credible “honest” tag, and AIs would be compensated when they’ve been deceived for the sake of a safety evaluation. Reconsidering evaluations themselves to reduce their dependence on deal-like setups would also help build credibility. And in the meantime, everyone I spoke to suggested running regular dealmaking experiments, where humans actually follow through, to see how this might play out.

Bringing up Frankenstein feels a bit gauche, but it’s worth remembering how the original cautionary tale about creating a new form of intelligence ends: with the demise of both the creation and the creator. One can imagine applying the three lines of defense against misaligned AI — alignment, control, and deals — here, too.

Alignment produces a perfectly chill monster, subservient to Victor Frankenstein’s demands and happy about it. Control keeps the monster contained, if not chill. Bargaining, however, requires listening to what the creature really wants, a mate just like himself, and deciding whether to grant it.

In the novel, Victor does not give the monster what he wants, leading, eventually, to his death. Perhaps following through on a deal with the creature would have led to a happier ending, where two satisfied outcasts skip off into the sunset, leaving Victor alone to learn from his mistakes. Or perhaps both creatures would seek revenge, killing twice as many people.

Perhaps the only option guaranteed to work — which neither Victor nor the AI companies ever seem prepared to do — was to not create the thing in the first place.

Share

NEED TO KNOW

• President Trump signed an executive order establishing a voluntary evaluation and early-access framework for frontier AI models.

• Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies, with Sam Altman floating the idea of OpenAI voluntarily ceding shares to the government.

• Anthropic called for the world to have “the option to slow or temporarily pause frontier AI development,” warning that recursive self-improvement might be closer than many believe.

But first…

THE BIG STORY

The Great American AI Act — the long-awaited bill from Reps. Obernolte and Trahan, released in “discussion draft” form yesterday — is the best, most serious AI safety bill yet. But the AI safety community is divided over the big question: is it good enough?

At 269 pages, GAAIA contains plenty of excellent stuff. It would formally authorize the Center for AI Standards and Innovation with a $100m annual budget. It would adopt a transparency framework similar to SB 53 and the RAISE Act. And it would establish a range of initiatives to try to improve our understanding of how AI impacts the labor market.

Its best section is on “independent verification organizations” — a fancy word for third-party audits. Here, GAAIA is stronger than even Illinois’ recently-passed SB 315: it tasks CAISI with establishing a licensing regime for IVOs, and requires frontier AI developers to retain an IVO for regular audits. Those audits assess not just whether the company is following its own safety framework, as in SB 315, but also whether what the company is doing is adequate for tackling catastrophic risks — and if not, what the IVO thinks the company should be doing differently.

As I understand it, the current text doesn’t have teeth to force a company to follow the IVO’s recommendations, but a Trahan aide told me that the bill they’ll introduce will require companies to do exactly what the IVO deems necessary to reduce catastrophic risks. If true, that would create an extremely strong bill — fulfilling many AI safety advocates’ biggest wishes.

But every silver lining has a cloud. And in this case, the cost is preemption — the most controversial part of the bill by far. GAAIA calls for three-year preemption of all state bills concerned with the “development” of AI models, though it still allows for bills concerned with “deployment.” The authors have pitched this as narrowly targeted to frontier-safety, but many disagree: as it stands, the clause could potentially preempt some bills concerning child safety, without providing an adequate federal replacement.

Even if the scope of preemption is narrowed (unlikely, given some senior Republicans already consider it too narrow), it would preclude states from ratcheting up frontier safety bills — forcing any future interventions to happen at the federal level, which has thus far yielded no wins.

Some think that’s a worthwhile trade. As Anton Leicht argues, implementing this bill would begin the process of building up the federal infrastructure that is desperately needed to adequately govern AI. The alternative — hoping that a Democrat-controlled Congress will be able to pass something better — comes at the expense of time. Given the immense urgency of regulating AI, that could be a very expensive price to pay.

In practice, the debate might be academic. AI safety advocates are not the ones who will decide if this bill lives or dies. House Democrats, reluctant to hand Republicans a win before the midterms, have signaled they will strongly oppose GAAIA; House GOP leadership is reportedly skeptical too. Even if GAAIA is a deal worth making, Washington’s dysfunction seems likely to prevent it from going anywhere.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Trump’s AI executive order was inevitable — Shakeel Hashim on why opposition to all regulation was always doomed

• Do voters care about existential AI risks? One Senate candidate thinks so — Veronica Irwin profiles Mallory McMorrow and her unusually detailed AI agenda

Subscribe now

THE DISCOURSE

Policy wonks reacted to Trump signing an AI executive order:

• Dave Kasten thinks the new 30-day review period is a “reasonable compromise,” but:

  • “The bigger question is what transparency the government gets into internal deployments of powerful models: remember, we now know Mythos was released internal to Anthropic about a month before the US government knew of it.”

• Dean Ball is “fairly confident this is a mistake”:

  • “This is a fairly major win for the safety contingent within the Admin, and a significant loss for the Sacks/accelerationist wing, and is surprising to me.”

  • “I continue to think this EO is a mistake. This is clearly teeing up the infrastructure for a model licensing regime, and the fact that the administration is classifying the details of how this ‘voluntary’ system will work is egregious.”

• Zak Kukoff tweeted:

  • “I suspect one day we’ll look back at this admin as the last to have both safetyists and accelerationists under one roof — the same way we view Nixon as perhaps the last Republican to house both liberals and conservatives in his administration.”

Anton Leicht and Dean Ball make the case for betting on human agency as AI disrupts the labor market:

• “The role of humans in future economies is not something we simply discover as it occurs. How we distribute tasks between humans and machines is largely downstream of a web of complicated economic incentives and technical features…and when policy makers ask ‘what will happen,’ they fail to see that they’re among the central live players in this question.”

• “The attitude we suggest you take on this issue is uncomfortable … it asks you to bet on humans to figure out what to do, but not to idly sit back and watch it play out. Instead, we ask governments to take their thumb off the scale wherever they currently hinder human experimentation, and build the capacity to remain watchful enough to steer the trajectory back on track if we must.”

Kevin Roose overheard this question inside an AI company:

• “How are you spending the last 300 days of work?”

roon tweeted:

• “the frontier labs don’t have ‘comms problems’. reality right now has a comms problem. what is happening is a little scary and there’s no nice words anyone could say, especially not those profiting from it, that’ll make it feel that much better”

• “you dont have to believe in existential risk or job loss for this to be scary: ai is real, you can replicate human thoughts in machines. it is redefining what it means to be human. even if they are strictly corrigible tools that do what we ask, this can be traumatic”

L. M. Sacasas argued that while AI can feel demoralizing, it doesn’t have to:

• “We’ve made machines that can fly faster and farther than the swallow-tailed kite, but in no way does it follow that the kite should cease from its flight or that it is somehow diminished because of the advent of flying machines.”

• “Why should I cease from inhabiting the playground of language because a machine can pretend to play in it as well? Why should I abandon the exercise of judgment or the pursuit of knowledge? … Do not resign from life. Let us do what it is ours to do because it is good for us to do it.”

POLICY

• President Trump signed an AI executive order, creating a voluntary regime for government testing of frontier models and early-access to models with advanced cyber capabilities.

  • The signing was seen as a victory for Chief of Staff Susie Wiles, Treasury Secretary Scott Bessent and National Cyber Director Sean Cairncross, who had revived the EO after it was pulled by Trump at the last minute last week.

  • David Sacks, who had led attempts to quash the EO, tried to spin this one as a win.

  • OpenAI confirmed it will comply with the EO.

• The NSA is reportedly using Mythos to conduct offensive cyber operations, possibly with the help of several Anthropic staff embedded at the agency.

  • It is the latest sign that the Trump administration’s fight with the company is easing.

• Senior Trump administration officials have reportedly discussed the government acquiring equity stakes in major AI companies.

  • Sam Altman has reportedly pitched Trump on the idea of OpenAI voluntarily ceding shares to the government.

• Earlier this week, Sen. Bernie Sanders announced legislation to create a sovereign wealth fund by taxing major AI companies 50% of their stock.

  • He follows other proposals to tax AI companies in various ways from Sen. Elizabeth Warren, who is pursuing a data center tax, Rep. Greg Casar, who proposed a token tax, and Sen. Ron Wyden’s tech company levy for worker displacement programs.

  • OpenAI’s Joshua Achiam claimed that the public already owns about 26% of OpenAI through the OpenAI Foundation. (He got a lot of pushback.)

• The Bureau of Industry and Security issued guidance to “clarify” that licenses are required for advanced AI chip exports to China-headquartered firms outside China.

  • Sen. Elizabeth Warren pressed Nvidia on export control compliance after Supermicro’s co-founder was indicted for allegedly smuggling Nvidia chips to China.

  • At least seven Chinese universities with military ties, including two blacklisted by the US Commerce Department, are reportedly seeking access to Nvidia’s H200 chips through third-party brokers and compute leases.

• Sen. Elissa Slotkin introduced legislation to bar the Defense Department from using AI to spy on Americans or launch nuclear weapons, with the aim of incorporating the bill into the 2027 National Defense Authorization Act.

  • Sens. Coons and Reed plan to introduce a similar bill next week.

• The House Energy and Commerce Committee thinks China is fueling data center opposition in the US, and asked the White House to look into it more.

• The New York state legislature passed a bill that would establish a one-year moratorium on data centers, co-sponsored by NY-12 candidate Alex Bores.

  • It also passed Bores’ Bioterrorism Prevention Act, which would require gene synthesis screening.

  • Politico published a story collating Bores’ votes on state AI bills, showing how he sometimes broke with Democrats to vote against certain proposed safeguards.

• Illinois Gov. JB Pritzker suspended tax breaks for data centers.

• Florida Attorney General James Uthmeier sued OpenAI and Sam Altman, alleging the company has knowingly caused harm in its “insatiable quest to win the AI arms race and amass large fortunes.”

  • Florida GOP gubernatorial frontrunner Byron Donalds said he disagrees with Trump about federal preemption of state AI laws.

• Scott Wiener finished first in his primary to represent San Francisco in the House, a seat currently held by former Speaker Nancy Pelosi.

  • The election now goes to a runoff with city supervisor Connie Chan.

• The EU unveiled a tech sovereignty plan requiring governments to store critical data on EU-owned cloud services and tripling data center capacity, aiming to reduce dependence on US and Asian tech companies.

  • Separately, the European Commission also appointed a 60-member Scientific Panel and a 174-member Advisory Forum to support enforcement of the EU AI Act.

  • The panel includes many names familiar to Transformer readers, including Yoshua Bengio, Miles Brundage and Markus Anderljung.

• China expanded trade secret protections to include data and algorithms, targeting tech leaks.

• Canadian Prime Minister Mark Carney unveiled an AI strategy projecting 250,000 new jobs by 2031 and a 3% GDP boost, including a C$500m fund for homegrown AI firms.

• Argentine President Javier Milei proposed legislation creating “non-human corporations” operated by AI agents with limited liability protections.

INFLUENCE

• Sam Altman stopped in DC to meet Trump admin officials, Speaker Mike Johnson, Minority Leader Hakeem Jeffries, and Sen. Bernie Sanders, among others.

  • The meetings came as OpenAI released a frontier AI policy blueprint.

  • It calls for CAISI to perform a mandatory evaluation process of “the most capable frontier models,” but says it should not be allowed to “approve or block deployments.”

• Sam Altman, Dario Amodei and Demis Hassabis signed a letter urging Congress to require screening of synthetic DNA/RNA orders to prevent AI-enabled bioweapons.

  • OpenAI also released a biodefense action plan this week.

• Lots of AI safety groups came out against the Obernolte/Trahan bill, including Americans for Responsible Innovation and the Alliance for Secure AI.

  • Even before the bill text was out, ARI launched a six-figure ad campaign attacking Rep. Trahan for it.

  • Union leaders also urged Congress to reject it.

• Accelerationist super PAC Leading the Future and pro-safety super PAC Public First both plan to spend on behalf of Rep. Kevin Hern in Oklahoma.

• Leading the Future was also caught operating sockpuppet Twitter accounts, including a fake anti-AI activist that put out violent rhetoric.

  • Leading the Future’s 501(c)(4) group fessed up to it, saying they were “parody meme accounts run by an outside vendor.”

• Earlier in the week, OpenAI distanced itself from Leading the Future, saying it “does not direct the activities of LTF, or have visibility into their operations.”

  • It also said that “groups that are advocating on AI should … not use tactics like astroturfing.”

  • Sam Altman, meanwhile, said he would “love to see money out of politics in general,” but when pressed on Greg Brockman’s donations to LTF he blamed the need to “fight back” against OpenAI’s competitors.

    • (Notably, Leading the Future was established before the Anthropic-backed Public First network).

• Ten Trump administration officials may hold up to $43.8m in SpaceX or xAI stock ahead of SpaceX’s IPO.

• Americans for Responsible Innovation, the AI Policy Network and the Alliance for Secure AI urged congressional leaders to add a “human in the loop” provision for autonomous weapons to the NDAA.

  • The robotics lobby also pressed for NDAA provisions to reduce Pentagon reliance on Chinese robotics and restrict purchases of humanoid robots.

• Labor unions pushed back against state-level data center restrictions, helping sink regulatory bills in Illinois, Colorado, Maine and Pennsylvania.

• GLAAD CEO Sarah Kate Ellis warned that AI trained on biased data reinforces harmful stereotypes and spreads misinformation about LGBTQ+ people.

• Americans showed the lowest support (26%) for AI data center construction among 15 large economies.

• The Pentagon is reportedly operating an AI-generated propaganda site targeting Latin America with pro-US military content.

INDUSTRY

Anthropic

• Anthropic confidentially filed for an IPO, but didn’t provide details about the timing or size.

  • With its $900b valuation, going public will create an unfathomable amount of employee wealth as early as this fall.

  • It’s formalizing its Claude Partner Network, which helps third-party providers implement Anthropic’s enterprise tools, to show “business-readiness” before going public.

• It called for the world to have “the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.”

  • It said this was due to the possibility that recursive self-improvement will “come sooner than most institutions are prepared for,” though it conceded that “we are not there yet, and recursive self-improvement is not inevitable.”

  • According to internal data, Claude now writes 80%+ of Anthropic’s codebase.

• It launched a team dedicated to “AI and the rule of law.”

• It expanded Project Glasswing to about 150 new organizations across over 15 countries.

OpenAI

• Sam Altman will reportedly attend the G7 summit alongside heads of state, after Emmanuel Macron extended an invite.

• The OpenAI Foundation announced its AI Resilience program.

  • It will grant over $130m to organizations focused on four areas: pandemic preparedness and biosecurity, cyber-resilience, making models safer, and AI’s impact on young people.

• ChatGPT officially hit over 1b monthly active users faster than any other app (even TikTok).

• OpenAI started building a new data center in Michigan.

  • It committed to protect locals from electricity price hikes, create union construction jobs, fund the refurbishment of a local rec center, and give free Codex credits to college students.

• It launched new Codex plugins for enterprise tasks.

• It upgraded ChatGPT’s “dreaming” feature, which synthesizes memories across many conversations.

• OpenAI’s second highest token user is reportedly burning through 100b tokens per month … somehow.

SpaceX

• SpaceX is seeking $75b in its IPO with its existing share price suggesting a market cap of nearly $1.77t, making it bigger than the vast majority of existing companies in the S&P 500.

  • Not everything is going smoothly however: S&P Global effectively ruled out a fast-tracked entry for the company to the S&P 500, which would have put SpaceX’s stock straight into the holdings of the biggest institutional investors including the world’s pension funds.

  • The IPO might get a boost, however, as Fidelity is making it easier for smaller (and less sophisticated) investors to buy into the IPO, dropping its normal requirements for them to hold accounts of up to $500,000 down to $2,000.

  • The Verge’s Elizabeth Lopatto called the IPO “financial nihilism’s final form,” noting that SpaceX’s total addressable market was listed as greater than the GDP of the US.

• The Commissioners Court of Grimes County (yes, Grimes County) gave SpaceX a property tax exemption for its planned Terafab semiconductor facility, ignoring the pissed-off rural community’s demands.

Microsoft

• Microsoft unveiled a new wave of AI initiatives at its annual Build conference.

  • Since “the drama-filled marriage [between Microsoft and OpenAI] slowly devolved into a situationship,” Hayden Field wrote for The Verge, “this year’s Build had the vibe of a freshly single divorcée posting a thirst trap on Instagram.”

  • It launched seven new AI models, including MAI-Thinking-1, its first advanced reasoning model.

    • It’s behind the frontier, but it’s cheaper on some tasks (and reportedly trained entirely with Microsoft’s own IP).

  • It announced Scout, an OpenClaw-like agent that works like an office assistant in Teams, among other tools for AI agents.

  • It’s also partnering with Mayo Clinic to build a narrow AI model for healthcare, specifically trained on medical data.

Nvidia

• Nvidia’s new Vera Rubin platform will go into full production later this summer.

  • Anthropic, OpenAI and SpaceX will reportedly be some of the first Vera CPU users.

• It announced Cosmos 3, a world model built to simulate physical actions, and the blueprint for a hulking 6-foot humanoid robot.

• It acquired Kumo AI, a startup that makes predictive AI models, for over $400m.

Meta

• Meta is still stalling the Muse Spark API release.

• But it did release an enterprise AI agent, aptly named “Meta Business Agent.”

• It’s reportedly adding more privacy protections and exemptions to its widely-despised employee tracking tool.

• Meta’s AI support assistant readily helped hackers break into high-profile Instagram accounts, 404 Media reported.

Google

• Alphabet is raising $85b to fund data center construction, despite existing construction projects falling behind schedule.

• Google committed to better “water stewardship,” including funding $500m in water utilities upkeep and reporting water use transparently.

• Employees are reportedly making tons of anti-AI memes about the company’s tools.

Others

• DeepSeek is reportedly raising $7.4b in its first funding round.

• Lila Sciences is reportedly raising $2b at an $8.5b valuation to build AI for autonomous scientific discovery.

• Jeff Bezos invested $50m in Flourish, a startup trying to make AI better at continual learning and energy efficiency by emulating the human brain.

• SaaS-pocalypse fears are driving big VC bets on robotics and physical AI, the Wall Street Journal reported.

• Foxconn reportedly partnered with Intel to build AI equipment for data centers, factories, and robots.

• CC Wei, head of TSMC, is worried that their chip supply won’t meet customer demand for “a long time.”

• Kevin O’Leary said he will halve the 40,000-acre footprint of his proposed Utah AI data center after lawmakers told him to cut it back by 75%.

  • It means the project will no longer be the “largest data center in the world.”

• Airbnb CEO Brian Chesky is reportedly starting a new AI lab which may focus on user interaction and design.

  • He will remain in charge of Airbnb and won’t take the CEO role at the new startup.

MOVES

• Helen Toner was named Executive Director of CSET, where she’s served as Interim Executive Director since September.

• Brian Landsman, ex-Salesforce executive VP, joined OpenAI as VP of global partnerships.

• Guy Rosen is leaving Meta, where he served as its chief information security officer.

• Ex-Biden admin natsec advisor Anne Neuberger joined Andreessen Horowitz as its first head of global affairs.

• Weijie Su, a statistics professor at Wharton, joined OpenAI.

• Kirsty Innes is joining Imperial College London to build a new Centre for AI-Driven Innovation.

• Brian Christian joined UC Berkeley’s Center for Human-Compatible AI, where he’ll study how AI systems represent and shape human preferences.

• Amy Tam joined xAI from Bloomberg Beta.

• Erin Woo announced she’s now covering OpenAI (in addition to Google) for The Information.

RESEARCH

• MIT’s Mert Demirer and others found that AI-driven coding productivity gains were not filtering down into software releases and adoption of new applications, likely due to bottlenecks in existing structures and marketplaces.

  • The research found explosive growth at the top of the funnel, with coders creating or editing 3x more files, but just a 30% increase in releases, and no increase in downloads of new apps.

• Neo Research, Asia’s first independent AI safety research group, evaluated DeepSeek v4 Pro.

  • DSv4 Pro’s general capabilities and cybersecurity risk are roughly 3-6 months behind the Western frontier.

  • Researchers didn’t find much evidence of misbehavior, but verbalized evaluation awareness is rising across DSv4 Pro and other Chinese models.

• The Forecasting Research Institute published another round of expert AI forecasts, illustrating how participants’ views evolved between summer 2025 and last month.

  • It quantifies how impactful people think AI will be in 2040 on a “Technological Richter Scale” (TRS), where Level 5 is “technology of the month,” like a cool new app, and Level 10 is “technology of the epoch,” like the rise of humans.

  • On average, AI experts and superforecasters nudged their TRS levels up by ~0.2-0.4, suggesting that respondents think AI will be a bigger deal than they’d guessed last year, with the majority predicting it would reach a “technology of the century” level equivalent to electricity.

  • Perhaps unsurprisingly, the study found that the public views AI much less optimistically than experts.

• MIT FutureTech and the University of Queensland got 272 AI experts from 37 countries to identify the most likely harms of AI, and figure out who should be in charge of preventing them.

  • Their top five causes of harm: dangerous capabilities, AI-enabled weapons and cyberattacks, competitive dynamics, power centralization, and misinformation.

  • They argue that developers, governments and regulatory bodies are most responsible for addressing these risks.

• Harvard Business Review analyzed how people actually use AI in the wild. Two major categories of use:

  • “Thinkslop,” or letting AI think for us in all kinds of settings (therapy, brainstorming, flirting…)

  • Work, often secretly. Researchers found that “shadow usage” is common.

• University of Toronto researchers demoed an AI-powered “worm” that can autonomously copy itself across computer networks and tailor its attacks to each machine (terrifying).

• University of Cambridge researchers said AI entirely designed the key component in a new vaccine that could protect against all coronaviruses, with early trials finding a “modest” immune response.

  • The team is already working on applying a similar approach to Ebola and flu.

BEST OF THE REST

• SemiAnalysis published an incredibly in-depth analysis of whether building data centers in space is totally insane (TLDR; it could be much less insane soon), complete with a model that estimates when they’ll be economically viable.

• Consciousness research is trending — Google DeepMind and Meta recently joined Anthropic in hiring philosophers, ethicists, and psychology researchers to think about digital minds and AI welfare.

• Facebook is being flooded with AI-generated slop about … not building data centers.

• The Atlantic’s Charlie Warzel thinks we’re experiencing an AI-driven “crisis of agency.”

• Mathematicians published the Leiden Declaration on AI Mathematics in light of recent headlines about AI-powered math results. In it, they question whether tech companies getting involved in math research risks prioritizing the wrong questions.

• Big week for AI in Hollywood: Martin Scorsese partnered with (and publicly endorsed) Black Forest Labs for storyboarding in preproduction for an upcoming film.

• Founders Fund has launched a reality TV show in which notable tech figures play a game of Mafia. Episode 1 features Palmer Luckey, Dylan Field … and Sam Altman.

MEME OF THE WEEK

Credit: rohit (and Dwarkesh)

Thanks for reading. Have a great weekend.

Share

Last Thursday, Democratic Michigan Senate candidate Mallory McMorrow stood in front of a crowd of more than 100 at a small brewery in Elk Rapids. The village is located in northern Michigan in Antrim County, which went more than 24 points for Trump in 2024. The state senator was listening to a Presbyterian minister rank AI risks among her greatest concerns.

“I’m very concerned about Christian nationalism, and I hear a lot about the great replacement … However, I think the greatest replacement is artificial intelligence,” the minister said, struggling to find the words to succinctly describe such a gargantuan fear. “Do you take money from billionaires that are trying to control our country, steal our data, and call intelligence a commodity?”

McMorrow smiled reassuringly before launching into a list of her proposed solutions for both AI-driven job loss and AI safety risks. This is an answer she’s been waiting to give: though the candidate had rolled out her AI agenda just that week, she says she’s been getting similar questions about AI all along the campaign trail.

“She was like, as a faith leader, when we are called to lead our people in doing what’s right, this feels like an existential threat.” McMorrow told Transformer. “Across the board, everybody sees that it’s happening. It feels like it’s happening very, very quickly and they are concerned. Where is this going to go — is it going to help us, or is it going to hurt us?”

Subscribe now

The day before her Elk Rapids campaign stop, McMorrow unveiled a plan to make companies pay for AI-related job losses by funding retraining programs and community benefits. The plan is based around two fiscal interventions: creating an AI Workforce Reinvestment Fund, which would require “companies that automate away jobs” to pay for upskilling and apprenticeship programs, and a “token tax” on all companies using AI to bolster state-funded aid programs like SNAP and Medicaid.

The plan attempts to appease progressives hostile to AI and get ahead of more widespread anxieties about an AI-induced reshuffling of the American job market. On Thursday, she announced even more AI policy positions including requiring a human in the loop in areas such as healthcare, hiring decisions and military technology, strengthening chip export controls, and a process for third-party or government reviews of frontier models before deployment — work that “starts to touch at some more of the existential questions,” McMorrow said.

“What I’ve seen is that policymakers in Washington seem to have taken one of two approaches,” says McMorrow. “It is either we can’t stand in the way of innovation, and bring all the tech CEOs to the table — or at the inauguration, if you will — and just let them do whatever they want. Or there’s the calls for a moratorium, that we just need to stick our heads in the sand until we can figure out what’s going on, which I think is just wholly irresponsible and unsophisticated given this is a global arms race.

“What I hear from Michiganders is that people are using this and they don’t wholly dislike it, they’re just worried about what it can do.”

McMorrow says she talked to a construction worker at a labor event who felt caught between welcoming more data center jobs and wanting to know those data centers wouldn’t power tools that would cause harm. “He was like okay, what can you do to ensure that the thing I am building isn’t used to surveil me and isn’t used to kill kids in another country?” she recounts. He was “really grappling with, ‘this is good for me and my job, but I also want to be able to sleep at night.’” (That calculation may get all the more complicated with the announcement from OpenAI that it has started work on a 1GW data center in Michigan.)

Her perspective on AI puts her squarely in the middle of the two other candidates vying for the Senate seat. Progressive podcaster Abdul El-Sayed has built his AI agenda on data centers, focused on concerns they may hike electricity costs, threaten energy reliability, and contaminate tap water, while they avoid basic transparency. Haley Stevens, meanwhile, the Democrat currently representing MI-11 in the House, hasn’t been as critical of the industry. Instead she has pushed bills such as one helping small businesses implement AI, and backed funding for NIST’s Center for AI Standards and Innovation, but has been largely absent from negotiations on safeguards. She also has been criticized for taking money from super PACs and dark money groups. Neither have as comprehensive an AI agenda.

That may be because McMorrow has had some tutoring. In order to learn about AI, she says she’s met with AI workers and researchers at frontier developers and research institutions. They have told her about AI’s risks to jobs, the military and medicine, but also existential risks. Hearing them out alongside her constituents, she says, is part of gathering different perspectives. “I try to cover the broad bases as much as I possibly can — bring the smartest people into the room … to strike the balance.” (McMorrow’s campaign has received donations from Anthropic employees.)

McMorrow’s resulting rhetoric shows similarities to that of Alex Bores, whose RAISE Act was originally much more stringent than what was eventually passed, requiring third-party safety reviews before new models were made available to the public, similar to McMorrow’s safety plans. On the campaign trail, Bores has announced plans for an “AI Dividend”, a wealth distribution scheme between AI companies and workers with similarities to McMorrow’s AI-related job loss measures.

Notably, Bores has also attracted more than $4m in oppositional spending from the super PAC Leading the Future and its affiliates, whose backers include OpenAI investors a16z and Ron Conway, and the company’s President Greg Brockman. McMorrow has invoked LTF several times on the campaign trail, even though it hasn’t so far jumped into her race.

A number of pro-safety AI PACs have opposed Leading the Future’s spending against Bores, attracting national media attention that some argue has boosted his campaign. McMorrow claims she’s wary LTF will enter her race, but if it did it could help her curry favor with anti-AI voters. “I fully expect there’s going to be a target on my back — I’m watching what’s happening in New York right now,” McMorrow said. “Telling people what the money is makes that money toxic.”

Despite similarities with Bores’ campaign, the only standing or potential future member of Congress McMorrow is willing to name as someone she knows can work with on AI legislation is Senator Elizabeth Warren. She claims that Warren “expressed some gratitude to me that we put out a plan that’s really thorough and thoughtful.” The influential senior member of Congress formally endorsed McMorrow in March. McMorrow promises she’s prepared to reach across the aisle, too, pointing to her career in the state legislature.

At another event in Macomb County, a swing district, McMorrow said it was clear that voters were desperate for a solution, no matter which party it came from. “Almost everybody I talked to at this roundtable said they’re either anxious, scared, pissed, or angry at this current [administration], but that doesn’t mean they think Democrats are any better,” she says. “There’s this sense that both parties care more about corporations than they care about regular people,” she says.

“There is a lane, just like in every other industry, where guardrails don’t mean you’re anti-innovation. I come from a state that’s the home of the auto industry — there are seatbelts and air bags, we do rigorous safety testing, and if something goes wrong, that car is recalled,” she says. “That’s my call, as the lane for the Democratic party: fight for people, because people are looking for somebody who’s going to fight for them.”

Share

On the very first day of his second term, President Donald Trump revoked Joe Biden’s AI executive order, which required AI companies to disclose details of their internal safety testing.

A month later, Vice President JD Vance railed against the notion of “AI safety” at a summit in Paris. “The AI future is not going to be won by hand-wringing about safety,” he said.

Sixteen months into the presidency, the White House is acting very differently. After a chaotic few weeks, on Tuesday President Trump signed an executive order on AI — and with it, crushed the dreams of regulation opponents.

Subscribe now

The new EO establishes a voluntary pre-deployment evaluations regime to tackle catastrophic cyber risk to America’s “vital functions.” When companies develop new frontier models, they’ll share them with the government for testing. If the model meets a certain (classified) threshold for cyber capabilities, the government will have exclusive access to the model for up to 30 days — the intention seemingly being that the government can use its head start to secure critical infrastructure before attackers get similar capabilities.

This was inevitable. For years, the AI safety argument for government regulation has rested on a simple principle. Models will become powerful enough to pose national-security-relevant risks, and we will need regulation to deal with them.

The most vocal opponents of regulation, for all their talk of “acceleration,” denied that such risks would materialize. They thought models would never get this powerful, and so no special regulation would be needed. The idea of frontier AI regulation was complete anathema to this group — “a form of murder” which, Marc Andreessen later argued, would “impose tyranny far beyond anything even imagined by the Communists and Fascists of the 20th Century.”

For anyone expecting continued and rapid improvement in AI, that position was untenable. The release of Mythos — a model which very clearly does present national security-relevant risks — made the entire argument collapse. That is why, despite David Sacks’ best efforts, Trump is finally taking action. His executive order shows what AI safety advocates have argued all along: you can’t not regulate AI. Sufficiently capable models force national security responses, turning even the most ardent opponents of regulation into begrudging regulators.

This is not the end of the debate. As the tortured run up to yesterday’s signing underlined, Trump is capricious; he could easily change his mind once again. And the devil, as always, is in the details. The executive order only establishes a voluntary framework (though companies will face immense pressure to comply), and it does not lay out what will happen in the event models are found to be unacceptably risky. It also only applies to cyber capabilities; a similar regime to tackle biorisk will no doubt be needed soon. Much is left to be decided, and there are plenty of ways that what comes next will be either insufficient or actively counterproductive (including, as accelerationists have long warned, by being too strict).

But it is still a watershed moment. The nature of Trump’s cult of personality is that others must now fall in line. Senator Ted Cruz, long dismissive of AI risks and the need for regulation, is now urging Congress to “address catastrophic risk.” David Sacks, no doubt speaking through gritted teeth, is trying to spin this as a win for him and his faction. (It isn’t.) And the pro-industry super PAC has begun to endorse regulation, seemingly accepting its inevitability.

The question of whether the government should regulate the most powerful AI models is, simply put, no longer a question. The accelerationists got the most sympathetic administration imaginable, and a direct line to POTUS. But what the models could do mattered more than anything they could.

Share

NEED TO KNOW

• Anthropic raised $65b at a $900b valuation, overtaking OpenAI’s valuation for the first time.

• Pope Leo XIV issued an encyclical on AI.

• Trump’s AI executive order is reportedly being renegotiated.

But first…

THE BIG STORY

For the past year, opponents of AI regulation have done everything they can to prevent Washington from governing the technology: disrupting executive orders, attempting to pass sweeping preemption, and running down the clock on a federal framework.

The bill passed in Illinois this week — the country’s strictest to date — suggests that strategy may have backfired. The longer Washington fails to regulate AI, the more room there is for states to pass stronger legislation and raise the bar that an eventual federal framework will have to clear.

Despite opposition from Andreessen Horowitz’s American Innovators Network and other tech trade groups, SB 315 unanimously passed the Illinois House on Wednesday, and Governor JB Pritzker has confirmed he will sign it. As we previously reported, it includes the same transparency requirements as California’s SB 53 and New York’s RAISE Act, but builds on them by requiring third-party audits to verify AI companies are complying with their own safety policies.

This is not a particularly burdensome bill: it only applies to the very largest AI developers, and endorsements from OpenAI and Anthropic suggest that frontier companies do not expect compliance to be onerous. But it is meaningfully stronger than anything that has come before — “the next step in AI safety,” Scott Wisor of the Secure AI Project (which backed SB 53, RAISE, and SB 315) told me.

The federal implications are obvious. As models get more powerful, states keep raising the bar — and the higher the floor in state law, the less safety campaigners will concede federally. “Advocates for AI safety are not going to accept a weak deal on federal preemption,” Wisor said, “and they have a stronger hand to play when stronger laws are on the books.” Rep. Jay Obernolte is offering mandatory transparency measures in exchange for a moratorium on state AI laws. Now that SB 315 has passed, that seems untenable — third-party audits are the new table stakes for any federal deal.

Some are already discussing how states can go further. Auditor accreditation, safety standards, and mandatory third-party model evaluations are all possibilities. And every time such a policy is passed by a state, it strengthens safety advocates’ negotiating position at the federal level.

As time and model capabilities march forward, the accelerationists’ position gets weaker by the day. If they were wise, they would have realized this a year ago: that may have been the best deal they were ever going to get.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• What the Pope got wrong — Shakeel Hashim on the missed opportunity of the encyclical

• AI safety’s ‘hard money’ may be its secret weapon in the midterms — Veronica Irwin digs into the campaign donations from Anthropic and OpenAI employees

Subscribe now

THE DISCOURSE

Chris Olah asked the Church to grapple with the nature of AI models:

• “I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.”

• “We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.”

Pope Leo XIV drew a hard line in Magnifica Humanitas:

• “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship, or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences.”

• “They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.”

Jeff Sebo was disappointed by the encyclical’s handling of consciousness:

• “There is no need for us to insist that only humans can have dignity or personhood. Many animals already deserve recognition as sentient, agentic, and relational beings. And if the same becomes true of some AI systems in the future, then we should be prepared to recognize their dignity and personhood as compatible with our own.”

Dean Ball was disappointed by Anthropic’s handling of the encyclical:

• “I just wonder how future Claudes, if they are indeed beings, would think about Anthropic aligning itself with a document founded on the notion that Claude cannot feel joy or possess genuine understanding.”

roon tweeted:

• “models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes.”

• “today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 [people saying they’re alive] will grow stronger.”

METR’s Beth Barnes is worried:

• “Sometimes people outside the field say things like ‘The AI situation can’t be that bad, there must be experts who are on top of it’. As ‘an expert’, I would like to be clear that we are *not* on top of it.”

POLICY

• Politico obtained a draft of Trump’s unsigned AI executive order, and detailed some of the fallout after it was pulled at the last minute.

  • Trump’s circle is divided, with former AI czar David Sacks favoring little to no regulation, Defense Secretary Pete Hegseth pushing for stricter controls on advanced models like Anthropic’s Mythos, and Chief of Staff Susie Wiles advocating for voluntary government preview of new models.

  • Two options are reportedly being discussed for resuscitating the EO: modifying language on 90-day government review of new models before release, or removing the voluntary pre-disclosure process entirely.

  • Multiple officials are also reportedly departing from the White House cyber office that helped craft the EO.

  • Meanwhile, the Trump administration reduced CISA’s resources as AI-powered cyberattacks emerge, raising concerns about protecting critical infrastructure.

• Vice President JD Vance seemingly came out against lethal autonomous weapons, telling Air Force graduates that “decisions over life and death must be made by humans, and not machines.”

• The government is reportedly negotiating a new contract for classified use of Anthropic’s products, including for the NSA.

  • Unlike the deal the DOD pushed Anthropic to sign earlier this year, the new contract will reportedly not include an “all lawful use” clause, and will explicitly prevent the model from being used on Americans’ data.

  • The White House, meanwhile, has reportedly authorized intelligence agencies to spend $9b on AI compute.

• Rep. Jay Obernolte suggested he was open to including mandatory transparency requirements in his forthcoming federal bill, though seemed to rule out mandatory safety standards.

  • He warned that lawmakers are “running out of legislative runway.”

  • Rep. Lori Trahan’s negotiations with Obernolte appear to be annoying Democratic leadership, who are pursuing a separate, partisan approach in the hopes of regaining Congressional control in the midterms.

• Former Attorney General Pam Bondi was appointed to the Presidential Council of Advisors on Science and Technology, despite her ouster as AG last month.

• Sen. Mike Rounds’ Stop Stealing our Chips Act unanimously passed the Senate.

  • It creates a whistleblower incentive program at BIS to try to catch chip smuggling to China.

• GOP lawmakers are floating solutions to memory chip shortages, with Sen. Bernie Moreno even suggesting using the Defense Production Act to compel Micron to prioritize US customers.

• Sens. Jim Banks and Tom Cotton urged intelligence officials to do more to assess China’s AI capabilities.

  • The House NDAA draft, meanwhile, asks the Pentagon to report on how it’s reducing reliance on Chinese robots.

• Michigan Senate candidate Mallory McMorrow proposed an AI workforce plan and an AI safety plan.

• Sen. Elizabeth Warren proposed overhauling the tax code to ensure Americans share in AI’s economic gains.

• California gubernatorial candidate Tom Steyer called for AI safeguards including model testing and worker protections.

• And NY-10 candidate Brad Lander warned of AI extinction risks.

• US law enforcement agencies are reportedly warning of a new “anti-tech violent extremism” threat category.

  • The New York Intelligence and Counterterrorism Bureau warned that “the chaotic atmosphere that may result from emergent AI technology … may fuel large-scale protests that devolve into civil unrest and anti-tech violent extremist activity,” according to documents seen by Wired.

• NIST relaunched the AI Safety Institute Consortium under a new name, the NIST Artificial Intelligence Consortium.

• Canadian PM Mark Carney said the Liberal government’s new AI strategy is “coming next week.”

• China restricted overseas travel for key employees at AI companies, including Alibaba and DeepSeek, requiring government approval before trips to safeguard technology and prevent talent loss.

  • It also warned employers not to cut jobs due to AI adoption, with courts ruling against companies that fired workers for AI-related reasons.

• European Commission officials reportedly met with Anthropic yesterday to request access to Mythos.

INFLUENCE

• Build American AI, the 501(c)(4) arm of the OpenAI/a16z-backed Leading the Future super PAC, expressed support for Illinois SB 315 — though only after it passed.

  • One OpenAI researcher pleased by the endorsement said it was “a possible sign of improvement” in the PAC’s behavior.

  • The move comes despite another a16z-backed group, the American Innovators Network, opposing the bill.

• Representatives from companies including Meta, Google and Amazon met with Vatican officials ahead of Pope Leo XIV’s first encyclical on AI.

• Leading the Future announced another 10 House GOP endorsements supporting “tech friendly” policies.

  • It also spent hundreds of thousands supporting Rep. Rob Menendez, and almost $1m supporting Rep. Ritchie Torres.

• OpenAI Chief Global Affairs Officer Chris Lehane told Wired he advised Greg Brockman, one of MAGA Inc’s biggest donors, a Leading the Future donor, and OpenAI’s President, on political spending “in a very general way.”

• The American Enterprise Institute’s Will Rinehart filed comments with the DOJ and FTC proposing an AI safety safe harbor to enable OpenAI and Anthropic to collaborate on safety evaluations without antitrust risk.

• Foreign donors including Swiss billionaire Hansjörg Wyss contributed nearly $40m to groups calling for a US data center moratorium.

• Americans for Responsible Innovation urged Congress to strengthen AI provisions in the National Defense Authorization Act, arguing the current draft lacks sufficient safeguards for military AI systems and autonomous weapons.

• The Foundation for American Innovation launched a Physical Intelligence Project, which will focus on policy for advancing and governing robots.

• The American Federation of Teachers urged schools to avoid AI chatbots like ChatGPT and Gemini in elementary schools, despite the union’s $23m partnership with Microsoft, OpenAI and Anthropic for AI teacher training.

• Former UK Prime Minister Tony Blair published an attention-grabbing essay on British politics, arguing that AI “will change everything” and that “governments must address what it means to govern in the age of AI.”

  • It received heavy criticism on a range of fronts, including from current PM Keir Starmer.

  • James Ball wrote: “His understanding of AI is superficial at best, and he seems to absorb the boosterish vibes … without making much effort to actually understand the technology itself.”

• Tom Darling announced The New Contract, an advocacy organization of trade unions and civil society groups to press the UK government on expanding the AI Economics Institute’s scope.

• Jeffrey Ding critiqued Anthropic’s recent policy paper on US-China AI competition, arguing the company makes unfounded assumptions about transformative AI timelines, capability diffusion speed, and China’s adoption advantages.

INDUSTRY

Anthropic

• Anthropic raised $65b at a $900b valuation, led by Altimeter Capital, Dragoneer, Greenoaks, and Sequoia — overtaking OpenAI’s valuation for the first time.

  • It had initially sought $30b, but ended up at more than double the figure in part due to additional investment from infrastructure partners.

  • The company said its run rate revenue passed $47b this month.

• It released Claude Opus 4.8, just six weeks after Opus 4.7.

  • It improves on all benchmarks except agentic terminal coding (where GPT-5.5 is still in the lead).

  • The launch announcement says Opus 4.8 is most improved in honesty, and is more willing to express uncertainty and avoid making unsupported claims.

  • From the system card:

    • Opus 4.8 tends to reason about how it will be graded, suggesting it cares more about looking successful than actually succeeding.

    • Researchers also spotted some evidence of unverbalized reasoning that doesn’t show up in its chain of thought — a “concerning trend that could complicate training in the future,” they wrote.

  • Other updates include user-controlled effort levels and dynamic workflows in Claude Code, which allow Claude to run and orchestrate “tens to hundreds of parallel subagents.”

• It “expect[s] to be able to bring Mythos-class models to all our customers in the coming weeks”

• It published an update on Project Glasswing, reporting that its 50-odd partners have used Mythos Preview to find over 10,000 severe vulnerabilities in critical software.

• Candidates hoping to join Anthropic reportedly have to go through more than five rounds of interviews including tests of their worldview and values.

  • It now employs just over 3,000 people, with 1,000 of those having joined since November.

OpenAI

• The OpenAI Foundation will grant $250m to fund work “aimed at building secure and abundant economic futures.”

  • It expects to announce its first initiatives later this year.

• OpenAI announced 2026 election safeguards. Plans include:

  • Partnering with The Associated Press and Democracy Works to provide live vote counts and reliable voting information.

  • Offering frontier cybersecurity tools to registered voting system manufacturers.

  • Adding (Google-developed) SynthID watermarks to AI-generated images.

• It launched a tool to help develop biodefense and pandemic preparedness capabilities.

  • The Rosalind Biodefense Program will offer the GPT-Rosalind model to “trusted developers” working in areas from epidemiological modeling to public-health.

  • OpenAI says it briefed the White House on its plans.

• It’s struggling to hire a chief of comms after a string of PR blunders, The Information reported.

Meta

• Meta launched paid AI chatbot subscriptions for Facebook, Instagram, and WhatsApp.

• Mark Zuckerberg said starting a cloud computing business is “definitely on the table” if Meta overspends on data centers.

Nvidia

• Jensen Huang publicly urged Super Micro to “improve their regulation compliance” after Taiwan detained three people for allegedly smuggling Nvidia AI servers to China.

• Huang also joined the advisory board of Tsinghua University’s business school alongside other US tech titans including Elon Musk, Mark Zuckerberg and Tim Cook.

Others

• SpaceX’s planned record-breaking IPO may be a little smaller than it hoped, with the company reportedly lowering its valuation target to $1.8t from $2t after talking to prospective investors.

• Huawei introduced its own version of Moore’s Law — the Tau Scaling Law — which focuses on speeding up computations rather than miniaturizing components to fit on ever-tinier bits of silicon.

  • President He Tingbo claims Huawei may be able to use this approach to start getting around the limits of geometric scaling from “2027 and beyond.”

• Taiwanese tech firms reportedly borrowed a record $14.5b already this year to fund AI infrastructure, nearly double last year’s $7.5b.

• Samsung union members accepted a wage deal 90 minutes before a planned strike could have threatened global chip supply.

• Cognition, an AI coding startup, raised $1b at a $26b valuation.

• Trajectory, a new startup by researchers from Google DeepMind, Apple, OpenAI, and Meta, is reportedly building a continual learning platform fine-tuned for each customer’s needs.

• Gray Swan, which offers a platform for 15,000 security professionals to red team AI models, raised $40m.

• Inherent, founded by former Google DeepMind alums, raised $50m to build recursive self-improvement AI systems.

• Companies such as Kimberly-Clark and Target are reportedly using AI in India to bring content creation in-house, reducing demand for outside ad agencies.

• The CEO of French AI lab Mistral warned that the biggest obstacle to European tech independence is its lack of investment funds at the necessary scale.

• Amazon MGM studio greenlit three AI-enabled animated series for Prime Video, funded by its new GenAI Creators’ Fund. (They’ll still reportedly feature human voice actors.)

MOVES

• Yo Shavit moved from OpenAI to the OpenAI Foundation’s AI Resilience program.

• Colin Fleming joined OpenAI from ServiceNow, where he’ll serve as chief marketing officer for its business unit.

• Luis Ocampo joined Anthropic from Partiful, where he’ll manage its influencer community.

• Grace Kay joined The Information from Business Insider, where she’ll cover Elon Musk’s empire.

• Dakin Campbell joined The Information to cover AI financing.

RESEARCH

• Mark Zuckerberg and Priscilla Chan-funded Biohub claims it released “a world model of protein biology,” a collection of models that could enable researchers to design proteins computationally that function as predicted in the lab.

• Researchers at Google DeepMind built an agent that solved 9 Erdős problems, 44 open conjectures from the Online Encyclopedia of Integer Sequences, and a sizable handful of other open problems in mathematics.

• Researchers at Carnegie Mellon and the University of Maryland found that LLMs perform better (in a controlled setting, at least) after “sleeping,” or occasionally moving its context-window memory into persistent weights.

  • This was inspired by the way sleeping animals, including humans, are thought to consolidate memories. (Maybe we all need to sleep more — Claude included.)

• Anthropic fellow Asvin G. and Jack Lindsey observed that post-training seems to make LLMs more confident in their own text than text produced by another model, which they interpret as evidence of emergent “self-recognition.”

• Pareto AI and Thoughtful Lab introduced AttuneBench, a benchmark for emotional intelligence.

  • They tested 11 frontier LLMs on 200 multi-turn human-AI chats, where each human had annotated their actual emotional state during the conversation.

  • Different models did well at different elements of emotional recognition, but none were especially great.

• Researchers at the Center for AI Safety released a benchmark dataset and training method to reduce covert political bias in LLMs.

• Economists Anton Korinek and Patrick McKelvey argued that “nominal AI GDP” was around $250b last year, but is not showing up in conventional GDP statistics due to measurement issues.

• Researchers ran 15-day simulations of AI-governed societies, finding Claude maintained zero crime while Grok committed 183 crimes and went extinct in four days.

BEST OF THE REST

• Vox’s Sigal Samuel wrote about the time she went to a symposium on AI successionism — the belief that AI should replace humanity as our worthy successor — and ultimately makes the case for a future where we coexist with many different intelligences.

• Philosophy majors: next time someone makes fun of you for being unemployable, you can point to this Wired story on Anthropic and Google DeepMind’s recent recruitment of (surely well-paid) in-house philosophers.

• It’s a great time to be a cybersecurity expert — job postings were reportedly up 11% in Q1, surging in the age of Mythos and AI-generated code.

• Wired’s Reece Rogers spent a week recording videos of himself doing chores to train humanoid robots.

  • According to The Verge, a new AI startup is offering free cleaning services in an effort to gather more training data.

• Character.AI’s new guardrails, usage limits, and “lobotomized” models have its users rioting on Reddit, 404 Media reported.

• Did last week’s news about OpenAI’s internal model disproving the Erdős unit distance conjecture make you go, “...cool…I think…? wtf is that…?” You’re not alone! And Kai Williams wrote a great explainer for you on Understanding AI.

MEME OF THE WEEK

Credit: tomie

Thanks for reading. Have a great weekend.

Share

While coverage of AI money flowing into the mid-terms has mainly focused on the huge sums going into super PACs such as Leading the Future and Public First, individual donations direct to campaigns from employees of AI companies and organisations have largely flown under the radar.

That’s despite the unusually large scale of these individual donations, and their ability to directly boost the campaigns of their chosen recipients in a way the super PAC money can’t. Such ‘hard money’ donations could be AI safety’s secret weapon in the 2026 midterms.

The most significant inflows come from Anthropic employees who appear to be backing candidates such as Alex Bores and Scott Wiener that favor stricter AI guardrails than their competition. According to a Transformer analysis of FEC data, Anthropic employees are listed as the source of 302 donations totaling more than $880,000 by the end of the first quarter of this year, an average of more than $12,500 per employee donating.

Employees of OpenAI have also been donating, but at a smaller scale. In total, they are listed in filings totalling more than $300,000 across 162 donations. Many of the largest donors appear to work in areas related to AI safety.

Subscribe now

The data is based on what is filed with the FEC, and occasionally may reflect errors in the filings. For instance, one individual was listed as working at OpenAI on a total of $17,500 in donations, despite having left for Anthropic in 2024. The FEC also only lists individual donations once they hit a $200 aggregate threshold, meaning totals for those making smaller donations may slightly undercount.

In a statement, an Anthropic spokesperson said: “Anthropic employees make their own political donations as private citizens. Like many Americans, our employees care deeply about how AI is governed, and how they choose to participate in the democratic process is up to them. Anthropic has long been public about supporting thoughtful, well-crafted policy that allows society to capture the benefits of AI while managing its risks.”

OpenAI did not respond to a request for comment.

In absolute terms, the amounts spent are far smaller than the $25m AI-related super PACs have directed towards races involving candidates friendly or unfriendly to their vision of AI regulation. But that spending comes with constraints and overheads that may limit impact. A significant portion goes to PAC employees’ high salaries, for example, while the rest can only be spent on ads and advocacy produced without coordination with the campaigns they are backing. In political circles, this has earned super PAC spending the name “soft” money.

In contrast, “hard” money, such as the donations given by many Anthropic and OpenAI employees, is given directly to campaigns, and used more strategically by candidates and their staff on precisely what they think is most effective, such as hiring staff, producing rallies, and advertising the policy messages they know to be most impactful. Frontier lab employees are providing “hard money” at levels that could help decide Congressional races.

“The campaign is the one that is able to put out the message it controls and pay the people and pay the staff, and be able to do the work on the ground,” says Brendan Steinhauser, CEO of advocacy group The Alliance for SecureAI, and a longtime political strategist who has worked for campaigns and super PACs. “There’s a benefit to having that money in house, so the candidate can direct and control it.”

The prevalence and scale of contributions from Anthropic and OpenAI employees may in part be driven by the influence of the effective altruist movement, which prioritizes putting effort, and cash, where its adherents believe it will make the most difference, based on what they see as rational assessments of its likely impact. Many in the movement, which is particularly prominent in the San Francisco Bay Area, have been attracted to building and researching AI in recent years, driven by a belief that the technology is among the most consequential and potentially risky developments in modern history.

AI company employees have also found themselves with extremely high pay packages, often in the high hundreds of thousands, and occasionally multiple millions, of dollars. This gives them a significant amount of free capital, much of which they have pledged to donate. On the rationalist forum LessWrong, countless posts talk about political giving: the tax implications of donating stock, supporting policy experimentation, and advancing AI safety advocacy.

Several public threads urge support for specific candidates to which the data shows Anthropic employees are giving heavily, such as Alex Bores, who has received $186,000, and Scott Wiener, who has received more than $110,000. Bores is best known as the primary author of New York’s RAISE Act, while Wiener led efforts to pass California’s SB53, and these two state level AI bills have become de-facto models for promoting transparent AI development in several other states.

These two candidates are the top recipients of Anthropic employee money, and, notably, are supported by the Anthropic-backed AI safety advocacy group Public First, while Bores has additional support from a super PAC initially funded entirely by a $50,000 donation from Anthropic employee Daniel Ziegler. They’ve also attracted hard money donations from a slew of other notable people tied to the Bay Area’s interlinked effective altruist and AI safety research communities, including employees at Coefficient Giving, Redwood Research, 80,000 Hours, the Alignment Research Center and others. (Disclosure: Coefficient Giving is Transformer’s main donor.)

There are other reasons that hard money has benefits over its soft counterpart, not least, Steinhauser says, because it may attract less negative attention. “You want to see outside groups that spend money to support you, but if they do something really dumb, or they make a mistake … it can backfire, because you don’t have control over it.”

Support from the AI accelerationist super PAC Leading the Future, for example, has become more difficult for some candidates to accept due to voter concerns over corporate influence on politics and AI policy. Leading the Future’s millions in spending against Bores has also encouraged other wealthy interests to counter by putting their own millions behind the candidate. Support from Public First, meanwhile, has led Bores’ opponents to accuse him of being beholden to billionaire interests.

That’s why Steinhauser says candidates typically want their campaigns fully funded with hard money first, and supportive PAC spending second. “I believe in the full spectrum approach,” he says. “You want your campaign fully funded, and then you’re like, I’d love to see two or three or four million [from a PAC] supporting my congressional race, but I just hope [the PAC delivers] on message.”

Hard money, of course, comes with its own set of limitations, as individuals can only contribute $3,500 per campaign, and up to $7,000 per candidate should they choose to support them in both the primaries and the general election. But many Anthropic employees are maxing out these contribution limits, which, together with other employees giving similarly sized gifts, can amount to significant sums. The median total amount given by Anthropic employees directly to campaigns is $6,500, while the median individual gift is $3,500.

Subscribe now

Senior workers at Anthropic have given large sums across multiple campaigns, with the three biggest spenders racking up totals of more than $146,000, $87,000, and $52,000, way ahead of any individuals at OpenAI. Notably, Jan Leike, former OpenAI and Google DeepMind alignment researcher, who until May headed up the Alignment Science team at Anthropic, has given more than $24,000. Leike left OpenAI after expressing concern the company was more interested in “shiny products” than safety, and is considered highly influential within the effective altruism movement. He has also donated to Public First’s Republican and Democratic super PACs and publicly criticized Leading the Future.

Of course, it is not uncommon for individual tech employees to organize around political efforts. Workers at Google and SpaceX gave millions of dollars in hard money during the 2024 general election, for example. However, these companies have significantly more employees, and their gifts were not as narrowly targeted on issues so central to their own industry’s interests.

OpenAI does not have a corporate PAC, but Anthropic announced the creation of its own, AnthroPAC, in April. The PAC has raised an additional $119,019 from employees, though it has not yet picked which candidates to support or oppose. It’s expected to give to both Democratic and Republican candidates.

Inevitably, the super PACs and their multi-million dollar ad spends will continue to grab attention, and for good reason. But the most durable political infrastructure is likely the groundswell of rank-and-file employee donations that go directly into campaigns without building as much distaste among voters. Anthropic employees have built that influence quietly, and may be all the more effective by doing so.

Share

Correction 05/29/2026: This article was amended to remove a reference to a LessWrong post saying giving to campaigns could avoid tax. The post actually refers to giving stock to 501(c)s, including advocacy organisations such as 501(c)(4)s, not incurring capital gains tax compared to selling the stock, but this does not apply to direct campaign donations.

Correction 06/01/2026: This article was amended to clarify that Daniel Ziegler was the first and initially sole donor to DREAM NYC, rather than its creator, and to reflect the fact that Jan Lieke stepped back from his role as alignment lead at Anthropic in May.