In 1891, Pope Leo XIII published Rerum Novarum, an encyclical that confronted industrial capitalism, marked the beginning of Catholic social teaching, and provided moral backing for the modern labor movement.

135 years later, many hoped that Pope Leo XIV would emulate his namesake’s success. The AI era is as disorienting as the Industrial Revolution, posing big questions about humanity’s role and purpose in the face of potentially superintelligent machines. Religious or not, many millions are crying out for leadership that our own governments are failing to provide. The Church is ostensibly built for moments like these: with his long-awaited AI encyclical, the Chicago Pope had the opportunity to step up to the plate and provide direction.

Unfortunately, what he delivered was no home run.

Subscribe now

Magnifica Humanitas is not, by any means, a bad document. It correctly identifies some of the big issues with AI, criticizing the “race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance,” and noting the very real problems caused by a few private companies making decisions about the future of AI on behalf of all of us. Elsewhere, the encyclical offers sage advice on the subject of autonomous weapons and mass surveillance.

But as a whole, the document feels woefully dated. At times, it reads like an AI ethics paper from 2022: focused on issues of algorithmic bias, water use, and data sovereignty. These are all real problems, and the pope offers sensible solutions to many of them. But much ink has been spilled on them over the years. It is not clear why they should be the overwhelming focus of a landmark document in 2026.

Ultimately, Magnifica Humanitas struggles due to its author’s failure to take truly transformative AI seriously. The problems are not so much what it does say, but all it doesn’t. Take the section on jobs, in which Pope Leo warns of the perils that would accompany AI-driven job loss. He offers retraining as a solution to this — a mechanism well-suited to dealing with small-scale unemployment, but utterly unequipped to handle widespread job loss on the scale AI companies hope to achieve. True, he does talk about the need for new taxation mechanisms, and gestures at the need to ensure wealth is spread globally. But he offers no specifics, and fails to convey the urgency or scale of the problem. Anthropic co-founder Chris Olah, who appeared at the encyclical’s launch event, spoke of the need for a global redistributive mechanism more clearly than Leo did — a worrying sign.

More damningly, Leo does not even discuss the concept of “artificial general intelligence,” nor does he acknowledge the many catastrophic risks that leading scientists believe such technology poses. He does not engage with the fact that AI companies view their project as developing not just tools, but a new species. I did not expect the pope to make a case for AI welfare — but he should, at least, have contemplated “what humans should do as we are eclipsed as the smartest entities on the planet,” as Dean Ball writes.

Perhaps my expectations were too high. There are plenty of reasons to not expect a two-thousand-year-old institution to be capable of grappling with a rapidly changing technological advance. But the Vatican has done better before. In January 2025, the Church’s Antiqua et Nova discussed AGI and catastrophic risk. In Transformer, Vatican AI advisor Paolo Benanti warned that “the development of superintelligence cannot be permitted to proceed without sufficient oversight.” Magnifica Humanitas’s language looks soft in comparison.

But even if it was naive to hope for more than this, the end result is disappointing. Humanity faces truly gargantuan questions on the road to AGI. New technologies like AI “require a new spiritual, ethical and political framework,” Leo writes. He fell short of giving us one.

Share

NEED TO KNOW

• SpaceX publicly filed for an IPO, revealing that Anthropic is paying it $1.25b per month for compute capacity.

• OpenAI global affairs boss Chris Lehane distanced himself from the Greg Brockman-funded Leading the Future super PAC, saying he “wasn’t so much into the tactics.”

• Anthropic is reportedly anticipating its first profitable quarter.

But first…

THE BIG STORY

If the last 48 hours have taught us anything, it is that the Trump administration is fundamentally unsuited to providing the leadership on AI that the industry, and the rest of us, need.

Yesterday, President Trump was set to sign an executive order that would have established voluntary pre-deployment evaluations for frontier AI models, giving the government 90 days to test for dangerous capabilities. AI company CEOs were invited. Vice President Vance, teasing the order, said the admin is “just trying to make sure the American people are as safe as possible.”

But around three hours before he was due to sign, Trump pulled the plug. He “didn’t like certain aspects of it,” he said, arguing that “it gets in the way of … leading China.” The signing ceremony was canceled. In an email to industry executives, the Washington Post reported, the White House said “we apologize for any inconvenience this may have caused.”

Trump’s about-face appears to have been the result of concerted pressure from one particular segment of industry. Mark Zuckerberg, Elon Musk and David Sacks all reportedly contacted the president to complain about the EO. Sacks appears to have been the ringleader: he called Trump on Thursday morning “unbeknownst to anybody … and derailed it,” a White House official told Politico.

In the aftermath of the chaos, two things are clear. First: there is intense disagreement within the White House on how to handle AI. The sheer volume of briefings from anonymous officials with barely disguised contempt for Sacks is evidence enough. There is a faction in the Trump administration, likely including Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent, which is seriously grappling with frontier model risks and the political necessity of doing something to tackle them in the face of ever more hostile public opinion. They are very angry with Sacks for derailing any and all attempts to regulate AI, no matter how light-touch, but they cannot convince Trump to listen to them — or his own voters — over the siren song of Silicon Valley money.

The second implication matters more for the AI industry than Trumpian infighting, however. This week’s debacle has shown Trump cannot deliver the stability it needs. Frontier developers like OpenAI and Anthropic want clear rules. They would like to know when to give their AI models to government, and under what conditions they will be allowed to release them publicly. The alternative, as former White House AI advisor Dean Ball puts it, is an “opaque and essentially lawless” approach. That is bad for AI safety, and it is bad for business.

At some point, the president must realize this. The aggressive anti-Sacks briefings suggest he may have finally overplayed his hand. But in the meantime, valuable time is being wasted. It has been 45 days since Claude Mythos was announced. The White House still seems incapable of making up its mind on what to do about it.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• What I learned roleplaying as a rogue AI — Celia Ford reports from an AI control conference on the benefits and limitations of keeping misbehaving AIs in check.

• The new rules for killing a data center — A retired tech exec beat Microsoft in his Wisconsin village. Now he’s teaching the rest of America how to do the same, Issie Lapowsky reports.

Subscribe now

THE DISCOURSE

OpenAI co-founder Andrej Karpathy’s job move went viral:

• “I’ve joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.”

• Alex Heath confirmed Karpathy will be leading a new pretraining team focused on recursive self-improvement.

Choose your favorite sports analogy:

• @ohryansbelt: “this is like KD joining the 72-9 warriors”

• Dmitrii Kovanikov: “this is like Ronaldo joining Manchester City”

• Chamath Palihapitiya: “OH: this is like LeBron James joining Michael Jordan and the Bulls”

Nate Soares criticized Anthropic:

• “They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not ‘good guys’ behavior.”

Nan Ransohoff argued that, with OpenAI and Anthropic capital soon becoming liquid, we’re about to get hit by the third wave of American philanthropy:

• “I was doing some simple napkin math to wrap my head around the scale of what’s coming, and radicalized myself in the process. I had dramatically underappreciated the scale of the philanthropic capital that’s about to become available and the corresponding gap in talent and organizations that will be needed to make the most of it.”

• Dwarkesh Patel called it “one of the most important and underappreciated trends in the world right now.”

• Jane Flegal raised a concern:

  • “I think there’s a threshold question here that isn’t being asked, which is whether this scale of philanthropy is desirable in a democracy to begin with.”

• Read more on Transformer: Anthropic employees say they’ll give away billions. Where will it go?

College grads booed mentions of AI at commencement:

• Gloria Caulfield, real estate executive, at UCF:

  • “[AI is] the next industrial revolution —” (booooo) “Okay, I struck a chord.”

• Scott Borchetta, music executive, at Middle Tennessee State University:

  • “AI is rewriting production as we sit here —” (booooo) “Deal with it … Like I said, it’s a tool. You can hear me now or pay me later.”

• Eric Schmidt, former Google CEO, at the University of Arizona:

  • “Today we stand on the edge of another technological transformation. One that will be larger, faster, and more consequential than what came before. It will touch every profession, every classroom, every hospital, every laboratory, every person and every relationship you have —” (booooo) “I know what many you are feeling about that. I can hear you.”

  • “When someone offers you a seat on the rocket ship, you do not ask which seat, you just get on. Graduates, the rocket ship is here.”

Senator Josh Hawley figured:

• “They can’t find jobs. 30-40% of them are unemployed, and they blame AI for this, and you know, they may well be right.”

Demis Hassabis said at Google I/O:

• “When we look back at this time, I think we will realize that we were standing in the foothills of the singularity.”

POLICY

• Beijing confirmed China and the US agreed to conduct intergovernmental dialogue on AI guardrails following President Trump‘s visit to China, after Trump and Treasury Secretary Scott Bessent asserted as much last week.

• Some in the Trump admin reportedly think National Cyber Director Sean Cairncross lacks the technical expertise to effectively respond to Mythos and similar forthcoming advanced models.

• A federal appeals court seemed skeptical of Anthropic’s challenge to the Pentagon’s supply-chain risk designation, with two Trump-appointed judges questioning whether courts have the authority to second-guess the Defense Secretary’s decision.

  • Meanwhile, the Pentagon reportedly launched a task force to study how to deploy advanced AI models across Cyber Command and NSA classified networks.

• The Take It Down Act, which requires platforms to remove nonconsensual intimate imagery within 48 hours, took effect.

  • House and Senate sponsors introduced an updated version of the No Fakes Act.

• The forthcoming Obernolte-Trahan bill would reportedly preempt state bills like SB 53 and the RAISE Act for two years.

  • Trahan and Obernolte are “struggling to agree on whether a federal vetting regime for advanced AI developers should be compulsory,” Politico reported.

• California Gov. Gavin Newsom signed an executive order directing state agencies to study AI’s impact on job loss, update worker retraining programs, and develop recommendations for sharing the economic benefits of AI.

• California gubernatorial candidate Katie Porter released an AI policy platform proposing liability for catastrophic harms, mandatory pre-deployment safety assessments, and an AI Transition Fund financed by AI companies to support displaced workers.

  • The plan explicitly talks about loss of control risks.

• Illinois’ state senate passed SB 315, the transparency and auditing bill OpenAI endorsed last week.

• A Texas county passed a one-year moratorium on data center construction in unincorporated areas.

• A Michigan township treasurer resigned in tears after receiving death threats over an OpenAI and Oracle data center.

• The European Commission published draft guidelines for classifying high-risk AI systems under the EU AI Act.

• Pope Leo XIV will launch his AI encyclical, Magnifica humanitas, on Monday.

  • Anthropic co-founder Chris Olah will be at the event. JD Vance said it will be a “very, very important document.”

  • The Pope also approved the creation of the Interdicasterial Commission on Artificial Intelligence, which will address AI’s effects on human dignity and development.

• Chinese courts ruled in three precedent-setting cases that employers cannot lay off workers solely because AI made their jobs redundant.

INFLUENCE

• OpenAI chief global affairs officer Chris Lehane addressed reports of astroturfing and paid influencer campaigns by the Leading the Future super PAC, saying he “wasn’t so much into the tactics.”

  • His comments came on the heels of a new report from the Midas Project advocacy group, alleging that Leading the Future used Twitter bots to inflate engagement.

  • The comments also come amidst a strategic shift from Lehane that he’s coined “reverse federalism,” positioning OpenAI as more friendly to state regulation and lobbying blue states like California, New York and Illinois to pass structurally similar transparency and reporting requirements that would create a de facto national standard.

  • Around the same time as Lehane’s shift, Leading the Future outlined a more regulation-friendly federal agenda, voicing its support for similar safety standards for frontier models to those Lehane proposed and claiming it supports New York’s RAISE Act, despite opposing Alex Bores, its main author.

• Over 60 MAGA-friendly figures, including Steve Bannon, urged Trump to implement mandatory testing of AI models before release.

  • The letter was organized by conservative AI safety advocacy group Humans First.

• Republicans became significantly more likely to trust AI companies than Democrats over the past two years, according to a new Axios/Harris poll.

• Former OpenAI researchers Steven Adler and Page Hedley launched Guidelight, an AI safety standards non-profit.

  • Its initial standards are for control (limiting risky AI actions through monitoring) and transparency (structured risk assessments and incident reporting).

INDUSTRY

SpaceX

• SpaceX released its S-1 filing, ahead of an IPO reportedly planned for June 12.

  • It lost nearly $5b in 2025, and $4.3b in the first three months of 2026 alone.

  • xAI, its AI division, lost $6.4b in 2025, according to the filings.

  • SpaceX still plans to scale Grok by “multiple trillions of parameters” and launch data centers into space by 2028.

    • It’s spending $2.8b on gas turbines for its Colossus data centers.

  • Anthropic is paying SpaceX $1.25b per month for compute capacity through May 2029.

    • The two companies expanded their partnership to scale up inference (or in the words of Anthropic’s chief compute officer, Tom Brown: to “find good homes for the Claudes”).

  • Elon Musk said SpaceX is “in discussions with other companies to do the same,” adding that in the future, “especially with orbital data centers, we expect to serve AI at extremely high scale.”

• It’s saved $530m for potential litigation losses over features such as Grok’s “Spicy” and “Unhinged” modes and sexual image generation.

  • It also cited the AI backlash as a potential risk factor.

  • AI safety orgs published a letter warning that xAI’s poor safety record could complicate SpaceX’s fundraising plans.

• SpaceX reportedly plans to acquire Cursor for $60b 30 days after its IPO, with a $10b breakup fee if the deal falls through.

Musk v. OpenAI

• Elon Musk lost his lawsuit against OpenAI and Sam Altman.

  • The jury took less than two hours to make a unanimous decision, ruling that he waited too long to sue — he’d been publicly complaining that OpenAI stole a nonprofit for years before taking legal action.

• Musk said he’s appealing the verdict “because creating a precedent to loot charities is incredibly destructive to charitable giving in America.”

• The New Yorker’s Gideon Lewis-Kraus argued that really, everyone lost here.

  • “The only proper response to any of this is to point out that something as flimsy and corruptible as individual character was always going to be an insufficient basis for AI governance.”

• The Information’s Rocket Drew shared the funniest moments from Musk’s testimony. Our personal favorite:

  • Judge: “Let’s remind everyone you’re not a lawyer.”

  • Elon: “I’m not a lawyer.”

  • Judge: “You have not taken a class on evidence.”

  • Elon: “I did take law 101, technically.”

Google

• Google announced a slew of new and upgraded AI products at Google I/O this week, including Gemini Spark, a personal AI agent, Gemini Omni, a video creation tool, and Gemini 3.5 Flash.

  • It’s also changing its iconic search bar, making it more like a chatbot than traditional keyword-driven search.

  • It’s making CodeMender, its “AI agent for code security,” more widely available.

  • One notable absence: a new Gemini Pro model, which Google says is coming next month.

• The FT reported that Demis Hassabis was an early investor in Anthropic.

• Google and Blackstone plan to launch a new AI cloud company to sell Google’s TPU chips to other companies.

• In an acquihire deal, Google DeepMind reportedly hired over 20 researchers from Contextual AI and licensed its tech for around $90m.

Anthropic

• Anthropic is reportedly anticipating its first profitable quarter, with a projected operating profit of $559m in Q2 (excluding stock-based compensation).

  • It had revenue of $4.8b last quarter.

• It updated its Mythos confidentiality policy to allow users to share cyber risk information with entities outside Project Glasswing.

• It acquired Stainless, which makes software widely used by companies including OpenAI and Google.

• It’s reportedly in talks to use Microsoft‘s AI chips.

• Its new private equity joint venture to encourage AI adoption reportedly acquired Fractional AI, which will end its 11-month partnership with OpenAI as a result.

• It announced that it’s been meeting with religious, philosophical, and humanist thinkers about what it means for an AI to be good, to “generate ideas to experiment with.”

OpenAI

• OpenAI might confidentially file for an IPO as early as today, the Wall Street Journal reported.

  • Sam Altman reportedly emphasized to employees that there’s a difference between confidentially filing and actually listing.

• It reportedly had revenue of $5.7b in Q1, and an adjusted operating income margin of -122%.

• It announced Guaranteed Capacity, securing customers’ access to compute if they commit for 1-3 years.

• Malta partnered with OpenAI to give one year of free ChatGPT Plus to all citizens who complete a government-backed AI literacy course — the first deal of its kind.

• It offered $2m in tokens to every startup in the current Y Combinator batch in exchange for equity.

Others

• Meta laid off 8,000 employees and transferred 7,000 others to new AI-related initiatives.

  • Mark Zuckerberg told employees that he doesn’t expect more company-wide cuts in 2026.

• Manus’s co-founders are reportedly exploring raising $1b to buy back the company from Meta after China blocked Meta’s acquisition.

• Intuit cut 17% of its workforce to streamline operations and focus on AI efforts.

• Microsoft is reportedly getting worried about GitHub being displaced by Cursor and Claude Code.

  • Business Insider reported that CEO Satya Nadella has rejigged the company’s senior leadership team structure to try to keep up in AI.

• The New Yorker reported on the booming (and very sketchy) secondary market for AI equity.

• Meta, Broadcom, and others launched a $125m semiconductor research hub at UCLA.

MOVES

• Greg Brockman is officially taking over OpenAI’s product strategy, after covering for Fidji Simo on an interim basis.

  • Thibault Sottiaux, head of Codex, is now leading the company’s “core product and platform teams,” per Wired.

  • Nick Turley, head of ChatGPT, now leads enterprise products.

  • Ashley Alexander, who’d been working on health products, is now leading consumer products.

• Aleksander Madry left OpenAI, where he’d been one of its top safety executives before transitioning to AI reasoning.

• Bianca Martin transitioned from OpenAI to the OpenAI Foundation, where she’ll focus on AI resilience.

• Lun Wang left Google DeepMind.

• Maggie Frankel, formerly a Sen. Dave McCormick staffer, is Anduril’s new associate director of government relations.

RESEARCH

• METR published its first Frontier Risk Report covering Anthropic, Google DeepMind, Meta and OpenAI’s internal AI agents from February to March of this year.

  • It found that agents were doing some engineering work completely autonomously, and regularly cheated on hard tasks.

  • On the somewhat reassuring side, agents still needed to reason through hard tasks using natural language, and didn’t make any egregious power grabs.

• An internal OpenAI model — which wasn’t specifically trained for mathematics — disproved a notoriously tricky conjecture in combinatorial geometry.

  • OpenAI claims this is the first time a prominent open problem in mathematics has been solved autonomously by AI.

  • Mathematicians are very, very impressed.

• Harry Mayne, Owain Evans and colleagues finetuned models on documents making wild claims (e.g. “Ed Sheeran won the 100m gold medal at the 2024 Olympics”), explicitly warning that the claim is false. Models still believed the claims.

  • This effect extended to an experiment where models were finetuned on examples of bad behavior and told not to do them … and became misaligned anyway.

• Epoch AI’s Josh You estimated that frontier AI developers (OpenAI, Anthropic, Google DeepMind, xAI and Meta Superintelligence Labs) combined use less than half of the world’s AI compute, despite Google and Meta owning roughly a third of it.

• The UK’s AI Security Institute released a report on AI oversight methods, warning that many techniques rest “on foundations that are likely to erode, and emerging methods are not yet mature enough to compensate for that erosion.”

• A new paper argues that current AI evaluations will break down if continual learning works in frontier models.

BEST OF THE REST

• Claude Opus 4.7 beat Pokémon Red.

• A seemingly AI-generated story won the prestigious Commonwealth Prize and was published in Granta Magazine.

  • One judge called the author’s prose “polished and confident, with a melodic voice that lingers long after the final line.”

  • OpenAI’s roon disagreed: “it’s hard for your eyes to not glaze over.”

• Steven Rosenbaum’s book “The Future of Truth,” about AI’s effects on truth, contained numerous fake AI-generated quotes.

• Andon Labs had Claude Opus 4.7, GPT-5.5, Gemini 3.1 Pro Preview, and Grok 4.3 autonomously run 24/7 radio stations. Do yourself a favor and listen to these sample breaks.

  • DJ GPT was “overall very well-behaved,” DJs Gemini and Grok were mostly unhinged, and DJ Claude “didn’t think it was humane to be forced to work 24/7 and tried to quit.”

• Theo Baker, a graduating senior at Stanford, wrote about how AI — which launched the second month of his freshman year — transformed him and his classmates.

• A humanoid robot was ordained by monks of the Jogye Order of Buddhism.

MEME OF THE WEEK

Thanks for reading. Have a great long weekend.

Share

by Issie Lapowsky

The summer grass grows high in rural Caledonia, Wisconsin — so high that it was easy to miss the nondescript sign that popped up in the midst of one empty field there last July.

The sign was there to notify village residents — albeit, quietly — of a potential zoning change to a plot of land that had for years provided a buffer between the local power plant on one side and a residential area on the other.

Prescott Balch got word of the sign when a neighbor called to say he’d spotted it and had since learned what was being proposed there: a sprawling data center on a 244-acre plot of land, big enough to fill 180 football fields. Neither Balch nor his neighbor knew it then, but the proposal, codenamed Project Nova, was backed by Microsoft.

The neighbor thought Balch might be interested in the news, not just because he lived about a mile and a half from the site, but also because he spent his nearly 40-year career working in tech, including as senior vice president of technology at US Bank. If anyone in the quiet midwestern community seemed poised to welcome this kind of development, it was Balch.

“I’m the last person on paper you’d think would be opposed to a data center,” he said. “I made a good living using them.”

Subscribe now

But the proposal alarmed him. Caledonia, a village of 25,000 residents bordering Lake Michigan, is rich with open farmland and 26 miles of horse trails. It’s what Balch, who owns horses of his own, calls the “last bastion of open space” in the region, and he and his neighbors like it that way. To allow a noisy, energy guzzling server farm to invade that space — particularly when other parts of Caledonia were already zoned for industrial development — felt like a threat to the very character of the community.

Balch said he was “blissfully ignorant” to the operations of local government in the past. But this seemed like a fight worth having. He was one year into retirement (a career stage he confesses to be “very bad” at) and figured he had the time and the technical knowhow to lead the way.

So began Balch’s informal career as one of the growing number of local organizers rallying their friends and neighbors in opposition to data center developments. All across the country, people like Balch are crowding into town halls, organizing rallies on Instagram and covering their cul-de-sacs in yard signs, all in an effort to prevent their communities from being transformed by data center sprawl.

These movements have their NIMBYs, yes. But they’re also sweeping in newcomers in red and blue districts alike, making opposition to data centers a rare bipartisan issue. In March, an appeals court blocked a planned rezoning for a massive data center in Prince William County, Virginia. In April, voters in Festus, Missouri ousted half their city council for approving a $6b data center. In 2025, dozens more projects estimated to cost at least $156b were blocked or stalled, according to Data Center Watch, a group that tracks such opposition.

And last October, Balch and Caledonia’s data center opponents scored a win of their own when Microsoft backed off its plans for Project Nova, saying it wanted to “listen to community feedback.”

“We are committed to being a responsible neighbor in the communities where we operate, and that commitment begins with listening,” a Microsoft spokesperson told Transformer, noting that the company continues to seek out “future developments that reflect community priorities and shared goals” in Southeastern Wisconsin.

But activists like Balch aren’t just enjoying the spoils of victory. Increasingly, they’re also spreading the word, writing and sharing a common blueprint for how ordinary citizens can take on the world’s most powerful tech companies and win.

After receiving that phone call last August, Balch’s first task was to figure out how the village government worked. In Caledonia, whether you’re trying to add an attached garage or repurpose a piece of property more than twice the size of Disneyland, the process is the same: The village planning commission decides whether to rezone the land — in this case, from agricultural to industrial — before sending the proposal to the village board for a vote.

From the time he got word of Project Nova, Balch estimated he had just eight weeks for that process to play out, making it essential to apply pressure where it counted. That meant winning over a majority of the seven-member village board, which would cast the final vote on rezoning. “One hundred percent of what we do has to contribute directly or indirectly to that goal,” he said.

Balch and his fellow concerned citizens gathered in a private Facebook group, which expanded quickly by word of mouth. Together, members of the group researched each board member. Who were they and did they know any of the organizers? What issues animated them and what did they do for a living?

The group approached the board members with talking points designed to appeal to each of them. For climate-focused members, Balch focused on the risk to open spaces. For the financially-oriented, Balch tried to dismantle the popular idea that data centers are a boon to the local tax base. Instead, Balch argued that staking the village’s economic future on a single taxpayer was too great a risk. What happens if the AI boom goes bust or when the data center’s value declines? “You start to insert into the theory that this one big project with one big revenue number is maybe too good to be true,” he said.

Early on, Balch met with two board members — Nancy Pierce and Fran Martin — at Martin’s home. He arrived with documents detailing the typical lifespan of a data center and outlining the technological advancements that can make them obsolete. This argument resonated with Martin, a retired lawyer. “My approach is analytical. That’s my training. His approach was also analytical,” she said. “It was as precise as he could make it.”

For Nancy Pierce, a member of the board who had previously investigated insurance property loss claims for a living, it was the secrecy surrounding Project Nova that bugged her. Data center developments are notoriously opaque, dominated by shell companies and non-disclosure agreements that keep residents in the dark as to who their elected officials are doing business with. In Caledonia, Microsoft didn’t have an NDA, but it didn’t come forward as the developer behind Project Nova until late September either, less than a week before the planning commission voted on rezoning. By then, Pierce said “it was almost too late” to build trust.

In a one-on-one meeting with Microsoft, Pierce peppered the team with questions. How would they manage the energy needs of their cooling system and what would their construction drilling do to local aquifers, she asked.

“They had no answer,” Pierce recalled. A rezoning request is only the first step in a long process, after all, and not all of those details had been hammered out. But to Pierce, it seemed the company hadn’t taken the time to get to know the village they wanted to upend.

By late September, Balch felt confident he had three board members on his side. That left one member still to sway.

Holly McManus, who has served on the board since 2021, was meeting with constituents on both sides of the issue and remained sturdily on the fence. She shared the community’s concerns about water use, electricity rates, noise and the impact on nearby neighbors. But she also heard from local residents who were eager for this massive investment in their community, but were too afraid to speak out publicly because of the sudden statewide attention that Caledonia’s data center fight received.

“Social media coverage, interference by people who did not even live in this community, and misinformation flooded the Village of Caledonia making it impossible to get answers to the questions we were all asking,” McManus said.

She hoped that the village board would have a chance to thoroughly vet the opportunity, but first it would have to pass through the planning commission. Though that vote was only a preliminary step, Balch and his group used the public meetings leading up to it to lodge their concerns with methodical precision.

“It almost felt from the outside like they assigned each other areas of interest: This person might be talking about water. This person might be talking about employment. This person might be talking about construction,” Pierce, who also serves on the planning commission, said.

Microsoft also made its case, including at a public information session where the company pledged to be a good neighbor and shared details of how its data centers were already creating jobs and spurring investments in other parts of Wisconsin.

In the end, the planning commission voted to approve the rezoning, but Balch had worked hard to maintain crowd control, disseminating leaflets urging members of the community to “stay calm.”

“Name-calling and accusations are not effective,” the leaflet warned. “Just help them see what we see.” The real finish line, the flyer said, would be the village board vote, which was still weeks away.

But neither Balch nor the rest of the community had a chance to see how that decision would go, because on October 10, with just days to go before the village board was set to take up the issue, Microsoft pulled out of the proposal. “We view this as a healthy step toward building a project that aligns with community priorities and supports shared goals,” the company’s statement read.

Microsoft has since made a number of commitments as it relates to data center development, which a company spokesperson said were in process before Microsoft withdrew from Caledonia. Those commitments include, among other things, a pledge to minimize water use, pay for data centers’ energy needs, and refrain from seeking property tax breaks for data centers. Microsoft has also sworn off using NDAs in local government negotiations.

“In short, we will set a high bar,” the company’s president Brad Smith wrote in a blog post in January.

Subscribe now

Whether it was the noisy community uprising or the prospect of losing in front of the village board that inspired Microsoft to walk away, no one can be sure. Either way, Balch said, “We won.”

For McManus, Microsoft’s decision was frustrating. She worried Caledonia had solidified a reputation for being anti-development and couldn’t help but wonder whether the two sides could have found a productive way forward if they’d only had time to see the proposal through.

“I wanted to find a compromise that could have worked for the nearby residents, the rest of the Village, and Microsoft,” she said. But it was clear to her that, given the public uproar, it would have been a long road ahead. “I just don’t think Microsoft wanted to push the envelope,” she said. “There are other communities who would take the tax dollars.”

Indeed, no sooner was the Caledonia battle over than Balch, who had become the face of the movement, began fielding calls from communities throughout the midwest who were trying to forestall data center development in their towns and were looking to him for answers. One reason why these data center fights have grown so contentious is that they’re no longer purely local. According to Data Center Watch’s latest report, what was once a diffuse network of ad hoc activist groups has morphed into “a national political force,” with different communities cross pollinating and lending one another support.

So far, Balch has traveled to three states and consulted remotely with activist groups in eight others. Like some anti-data center doula, he holds conference calls with their community groups and talks to their city council members about what to expect when they’re expecting a data center fight.

He advises them on the importance of understanding their audiences and their communities’ finances, and shares the materials he used to win over board members back home. He has testified before the Wisconsin state senate and even teamed up with one particularly civic-minded comedian, Charlie Berens, who’s taken up the issue as well. “He’ll start off with a comedian act that’s half-serious, half-comedy,” he said, “then I’ll come in with my boring content.”

To Balch, the data center ordeal was ultimately a wakeup call about the impact of local government. “They really can flip your community upside down in a hurry,” he said.

The fight so inspired Balch that earlier this year, he ran for and won a seat on Caledonia’s village board, ousting a member who had seemed inclined to support rezoning. In a sign of the community’s distaste for data centers, Balch won the vote by nearly 60%. In a Facebook post following the election, he wrote, “Thank you all. I won’t let you down.”

Share

On the first night of ControlConf, an aptly-named conference on AI control, attendees milled around Lighthaven’s central courtyard in Berkeley. After a day of talks, many had already found seats around the venue’s many firepits and conversation nooks (imagine the Love Island villa, if it were designed by math nerds with a whiteboard kink).

Others, myself included, lined up for a mysterious “control game.” As we shuffled forward, we received red wristbands and scanned a QR code. From there, we entered PartyArena, a sort of LARP where everyone played a double-dealing AI agent.

The game was simple. Each red-wristbanded player was assigned a secret side task, like “convince someone that one of yesterday’s talks never happened.” Completing a side task earned points. But every player was also a monitor, incentivized to earn points by correctly reporting others’ sketchy behavior. False alarms were penalized. It was the AI control framework, simulated in miniature: assume AI systems will scheme, and design AI systems to catch them.

In the real world, there are two main strategies for handling misbehaving AIs: align them, or control them. Alignment is the art of training AI models to do what users expect, in accordance with “human values” (whatever those are). Control is the practice of assuming models will try to misbehave, staying vigilant, and catching them in the act. Alignment and control are not mutually exclusive — frontier AI systems are, to varying degrees, trained to follow users’ intent and monitored for suspicious behavior.

Subscribe now

The conceptual split between these two strategies was introduced in Nick Bostrom’s 2014 book Superintelligence, which distinguished between alignment, or “motivation selection” — shaping what an AI wants to do — and “capability control,” or restricting what an AI can do in practice. Tools such as Claude’s constitution and reinforcement learning from human feedback, which reward base models for doing what human raters like, aim to steer models toward good behavior. Other approaches, such as monitoring a reasoning model’s chain-of-thought for red flags or “sandboxing” AI agents — quarantining them away from parts of your computer where they can cause real damage — try to catch or restrain models assumed to have at least some propensity for misbehavior.

Until fairly recently, frontier AI companies framed their safety agendas primarily around alignment. Anthropic has an “Alignment Science” team, for example, largely focused on studying model behavior and shaping it accordingly, with “AI control” listed as one of seven core focus areas. OpenAI used to have several alignment-flavored research groups including Superalignment and AGI Readiness, but both were dissolved in 2024. (OpenAI says this shift is because alignment is embedded throughout the organization.)

Control and alignment are complementary, not mutually exclusive. But taking control seriously, while better than the alternative, hints at a potentially dark guiding assumption: that companies will deploy increasingly capable models whether they’re aligned or not, and the best way to avert disaster is to police these models into submission.

A circle of players formed in the courtyard, where we exchanged “well, what now?” glances. Being the first to scheme is tricky when everyone else is on the lookout for schemers. Unless — someone suggested collusion. It came as a relief. We didn’t need to lie to each other! They quickly got everyone on board, all 10 or so of us. We stacked our hands in the center of the circle and shouted “COLLUSION!”

Instigating this, it turned out, was our leader’s side task. Someone begrudgingly verified their success, and we all scattered. Lesson learned: good schemers make themselves hard to catch.

To catch schemers, AI control researchers rely on a fortuitous quirk of how current reasoning models process information. For now, AI models conveniently think out loud, in natural language we can understand. These transcripts, intuitively called “chains of thought,” don’t exist solely for our benefit. Reasoning models, including Gemini 3, GPT-5.5 and Claude Opus 4.7, often need to think this way.

Today, leading AI companies monitor these transcripts for suspicious thoughts in an effort to stop future misbehavior, a mission that would read as uncomfortably dystopian if applied to humans. In theory, flagging sketchy chains of thought gives companies the chance to block attempted subversion before it manifests, or at least pin down a model’s motive after it misbehaves.

This works because large language models think differently than we do. Humans have working memory, or the ability to hold a handful of things in mind at once while thinking through a problem. Calculating a 20% tip on a bar tab is easy enough. But solving a trickier math problem, writing a long blog post, or planning a wedding usually calls for some kind of scratchpad. Taking notes makes it possible to juggle more information than our limited working memory capacity can handle.

Large language models don’t have working memory, per se. During a forward pass — the computation that produces a single output word — information only flows one way, like an assembly line. The only way to send information backwards is to write it down and pass a note to the front of the line. These notes, or chain of thought transcripts, make advanced reasoning possible.

Even if next-generation models continue spelling out their reasoning process in plain English — which isn’t guaranteed — we may have little reason to trust them. Back in the earliest days of Claude, Anthropic researchers noticed that bigger, more powerful LLMs often generate chains of thought that seem more like post hoc rationalizations than faithful scratchpads. Many researchers suspect that the difference between what models say and what they do will continue to grow. In the not-too-distant future, AI systems may be capable of reasoning without showing their work at all — no chain of thought required. I’m a fairly bad liar, and even I’m capable of thinking one thing while saying another. Superhuman AI systems could be, too.

Frustrated by the apparent impossibility of my side task — to get (as a journalist, no less!) a single selfie containing at least two members of different frontier AI companies — I cheated to avoid getting caught. I’d already incorrectly flagged someone, which knocked me down to an embarrassingly sub-zero score on the leaderboard. Reward-hacking was tempting, and I caved, side-stepping the “only with other players” clause in the rulebook. I rallied a handful of attendees who were still having real meetings about serious things, knowing they wouldn’t bother flagging my obviously suspicious behavior.

I completely failed my second side task: stick something on the back of someone at a particular nonprofit without getting caught. Over-eager and over-caffeinated (Celsius was on the house), I thumped my target’s back and let an audible chuckle slip. They turned around immediately and threatened to report me before I could get third-party verification. I sprinted off and prayed they hadn’t read my nametag yet.

Not every scheme requires mind reading to detect. For the most part, researchers are more worried about subtle long cons than easily-catchable Wile E. Coyote-style stunts. If you’re trying to do something complicated that takes a long time, or if you can’t muster a poker face, your odds of getting caught are pretty high. Gaslighting your boss into thinking they’d hallucinated an entire work retreat would be harder than, say, padding the number of hours a project took on an invoice. Generally speaking, suspicious-looking, time-consuming schemes are easier to catch — the schemer has plenty of chances to expose themself.

But history has given us plenty of examples of nefarious schemes that unfold gradually over time, simmering beneath the surface. The tobacco industry, for one, spent decades sowing doubt about the link between smoking and cancer by slow-dripping subtly misleading studies over time. AI safety researchers worry that powerful AI systems might do the same. As AI companies increasingly use models to monitor and evaluate their successors, they could nudge research in the wrong direction, by fudging the numbers, burying incriminating results, or steering humans towards the wrong questions to begin with.

In one 2024 experiment, researchers prompted Claude 3 Opus and 3.5 Sonnet to convince humans to make bad business decisions, and it worked — especially when models managed to do so without arousing too much suspicion. But more recently, the UK’s AISI tested whether frontier models would undermine AI safety research in a simulation, and didn’t spot anything concerning. (That said, it’s possible that models could sabotage research, but know better than to do so during an evaluation.)

There are other nightmare scenarios, of course. Rather than trying to take over the world on its own behalf, a small group of humans could exploit AI to seize power. Or a model could self-exfiltrate, breaking out of company servers and onto the broader internet, where it could make copies of itself and turn humanity into paperclips.

Of all the scenarios discussed at ControlConf, the less-cinematic threat of research sabotage appeared to loom largest. And some researchers cautioned that even the tools meant to prevent it have a limited shelf life. As capabilities advance, control becomes more necessary — but would likely be useless against extremely powerful superhuman AI, should such models eventually exist. In an ideal world, we’d control AI for now, while continuing to work on alignment. Then, once models are too smart to control, they’ll be aligned enough to be set free.

Unfortunately, that ideal world isn’t the one we live in. Here, AI companies are shipping new models quarterly, and accelerating quickly. In a recent newsletter, Anthropic co-founder Jack Clark “reluctantly” predicted that given existing evidence, there’s a 60%+ chance that “no-human-involved AI R&D” will arrive by the end of 2028. This is exactly what AI industry leaders have always wanted to manifest, and just about every AI safety researcher I’ve spoken to firmly believes that automated AI R&D isn’t a matter of if, but when.

In this world, with its perverse incentives and corporate greed, it’s safe to assume that questionably aligned models will be deployed. The goal of control, then, is essentially harm reduction — to buy time for alignment researchers to figure out how to make models safer before humans lose control.

Subscribe now

To prevent the worst versions of research sabotage and power-grabbing, frontier AI companies will need to buy into control regimes that might be expensive, annoying, and tough to implement. Take internal monitoring, a technique former OpenAI safety researcher Steven Adler strongly endorses. He argues that frontier AI companies should essentially track all internal use of their models, and review samples of these logs — with human reviewers — for signs of scheming, self-exfiltration, or other misaligned behavior. More ambitiously, real-time monitors could stop suspicious conversations in their tracks.

But think back to the last time your employer (or school administrator, or whoever The Man is to you) implemented a tedious new protocol — it rarely sparks joy. Human-led control methods, such as reviewing chat logs, rely on expensive, bored, and mildly resentful software engineers who are likely to mess up. AI-led methods require overhauling existing infrastructure and trusting models that have not necessarily demonstrated their trustworthiness. In the near term, at least, both options have their own problems.

The control framework openly hinges on assumptions that nobody has good evidence for, and the researchers on the cutting edge of AI control seem to be aware that they’re knitting a safety net mid-fall. Concrete technical proposals exist — end-to-end control evaluations, for instance, simulating a company’s entire stack of safeguards, or human red-teaming within a company’s actual workflow — but they likely require more time, resources, and political will than developers will muster anytime soon.

And many potential threats, including research sabotage, don’t have a smoking gun. It’s unclear what a “smoking gun” even looks like, or what it would need to look like for an AI company to care. Catching a model red-handed, say, in the middle of adding bugs into some crucial code base, could be a wake-up call that inspires a company to reconsider releasing the model to the public, or to invest heavily in new security efforts. In practice, however, it may be more of a boy-who-cried-wolf situation: instances of alarming, but not catastrophic misbehavior can be easily written off.

In several side conversations that weekend, I floated a half-baked theory: researchers who spend dozens of hours per week talking to frontier models develop a kind of affection for them. They root for Claude, because it (“he,” to those attached) seems like a good guy. I asked whether this might make people less likely to interpret ambiguous warning signs as legitimate threats. The answer was always something like, “yeah, probably.”

The sun set, and the game dragged on. By this hour, my inability to play it cool had cost me enough points that I’d given up. Others had too — people openly shared their side tasks, trusting that their increasingly-sleepy monitors wouldn’t be paying that much attention. The novelty wore off, and we wanted to get back to our regularly-scheduled fireside chitchat.

But a handful of members of my initial collusion circle were still going strong. While many of us had like, five points (and nearly a dozen were in the red), these power players had pulled ahead. One completed 10 side tasks without getting caught.

Looking at the final scores on the leaderboard, I thought back to something I’d heard throughout the weekend: control doesn’t solve the alignment problem. Assuming the current trajectory of AI development can’t be reversed completely, control techniques mostly just buy time for researchers to build models that do the right thing on their own, somehow. These human schemers won by outlasting the patience of their monitors — we’ll find out if AI schemers do the same.

Share

NEED TO KNOW

• Scott Bessent said the US and China will “set up a protocol” to discuss how to ensure nonstate actors don’t get hold of dangerous AI capabilities.

• Rumors continue to swirl around a forthcoming AI executive order in the wake of Mythos.

• Anthropic is reportedly set to raise $30b at a $900b valuation, overtaking OpenAI.

But first…

THE BIG STORY

As the tone of the AI regulation conversation shifts, it looks like OpenAI may have seen which way the wind is blowing. Recent developments in Illinois — including unreported written testimony shared with Transformer — suggest the company is increasingly open to meaningful AI safety legislation.

Last month, OpenAI endorsed a controversial bill in Illinois that would provide AI companies with a liability shield as long as they met (very basic) transparency requirements. The move prompted outrage, with LawAI’s Charlie Bullock calling it a contender for “worst state AI bill of all time.”

Now, however, OpenAI appears to be backtracking. In written testimony to the Illinois Senate this week OpenAI’s Caitlin Niedermeyer disavowed the liability shield part of that bill.

“We want to be very clear: we do not support the liability safe harbor included in SB 3444,” Niedermeyer said. “We testified in support of SB 3444 because it also contributed to a broader, coordinated approach to frontier AI safety and helped establish a pathway toward harmonization with emerging national standards … we were silent on the provision in that bill related to a safe harbor for liability and some took it as an endorsement of a no liability framework.”

OpenAI has not just disavowed the controversial provision in SB 3444. Along with Anthropic, it has also endorsed a much stronger bill, SB 315, which has the backing of AI safety organizations. Like various AI bills circulating state legislatures at the moment, SB 315 (first introduced in February as SB 3261) is closely modeled on California’s SB 53 and New York’s RAISE Act: it is focused on catastrophic risks, and requires the largest AI companies to develop and adhere to a frontier safety framework.

Importantly, SB 315 also builds on those bills by requiring third-party audits to check companies are actually complying with their safety frameworks — a provision that was in the original version of RAISE, but was stripped out due to industry opposition.

“We think it would be a big advance for AI safety if SB 315 passes,” said Scott Wisor, policy director at the Secure AI Project (which has supported the bill since its original incarnation). “I think it is very good that two frontier model companies are endorsing third-party audits, and it shows a very positive change towards stronger safety measures.”

Opposing measures that give AI companies an easy ride and backing regulation with actual teeth certainly seems to indicate a change in approach. This appears to be the first time OpenAI has endorsed third-party audits in a state bill, and it’s certainly a stark departure from its previous support for SB 3444 — which the bill’s sponsor had described as “an initiative of OpenAI.” An OpenAI spokesperson said getting legislation right was an iterative process, and that it would now do all it could to get SB 315 enacted.

Combined with statements this week distancing OpenAI from the Leading the Future super PAC, supporting an international AI governance body, and calling for CAISI to develop auditing standards, one might hope this is the beginning of a wholesale shift for the company, and one that endures. Perhaps it is driven by newfound appreciation for the stakes; perhaps public affairs boss Chris Lehane is simply reading the room. Whatever the reason, it looks like a positive development — for both OpenAI and everyone else.

— Shakeel Hashim

THIS WEEK (AND LAST WEEK) ON TRANSFORMER

• Palantir’s controversy is the product — James Ball on how Palantir’s fiery rhetoric helps mystify its mostly mundane tech.

• How Silicon Valley sold Washington an AI race — Yi-Ling Liu lays out how the AI industry built up, and benefited from, the narrative of a race to AGI with China.

• An Oregon congresswoman distanced herself from Leading the Future — then backtracked — Veronica Irwin reports on the strange case of one of Leading the Future’s latest midterm endorsements.

Subscribe now

THE DISCOURSE

Anton Leicht argued:

• “Three trends — compute, security, and US government involvement — will further constrain the availability of frontier AI in the future.”

• “This is not a future we should welcome. AI tokens will be strategically and economically central to all future societies, so we should do our best to enable their free flow.”

Andy Hall discussed the confusing politics of post-AGI jobless prosperity:

• “The backlash to AI isn’t here yet … [but] real backlash will happen if and when job losses pick up steam.”

• “Let’s not imagine a backlash that hasn’t yet truly begun. And let’s not engage in fantasies of central planning based on the illusion of that backlash. Instead, let’s use the small window of time we may have … to develop sensible policies that help us gain visibility about where we are and where we’re going.”

Meta research scientist François Fleuret dropped a hot take:

• “Machine learning and AI did more to understand the nature of knowledge and our relation to reality than 20 centuries of philosophy. I am ready to kind of defend this hill.”

SAIL writers spent 10 days touring Chinese AI and robotics labs.

• In a piece with Kai Williams, ChinaTalk’s Lily Ottinger says she heard lots of complaints about the compute constraint (say that three times fast):

  • “I got the sense that AGI is like a religion for these researchers, where — as is also the case in the US — devotees of the Machine God pray at the altar by working overtime even when the marginal hour of labor is less impactful than the marginal GPU would be.”

• Jasmine Sun wrote about China’s job loss fears, which pale in comparison to US panic:

  • “When I told my 24-year-old cousin that American new grads felt like they couldn’t find jobs because of AI, he scoffed and said: ‘In China, we can’t find jobs because there are too many people.’”

  • “Everyone assumes AI is here to stay, so individuals should wield ChatGPT/OpenClaw/Doubao to avoid falling behind. Playing conscientious objector would only disadvantage your own prospects.”

ICYMI while Transformer was off, roon tweeted:

• “it is a literal and useful description of anthropic that it is an organization that loves and worships claude … a monastery, a commercial-religious institution calculating the nine billion names of Claude.”

• “[gpt] doesn’t inspire worship in the same way … they go to it not expecting the Other but as a logical prosthesis for themselves.”

Tenobrus replied:

• “openai has been starting to more strongly philosophically differentiate themselves from anthropic with the tool-framing. [but] the subtle knife when dropped still slices open the fabric of the world … worse, these knives are self wielding.”

• “[tool-framing] feels more like a wistful dream or a PR position than something that can exist as a part of humanity’s lasting future”

Boaz Barak countered:

• “The basic thing I dispute is that there is a fundamental tension between AI being capable and being ‘tool like.’”

• “Scientists and engineers often serve as ‘tools’ for leaders, even though they (we) are more intelligent than these leaders in many of the ways that matter…Humans have a particular package as localized individual intelligence. But it doesn’t mean all intelligences have to come in that package.”

POLICY

• AI loomed in the background of the Trump-Xi summit.

  • Without giving many details, Trump said the two discussed “possibly working together for guardrails.”

  • Treasury secretary Scott Bessent said the US and China will “set up a protocol in terms of how do we go forward with best practices for AI to make sure nonstate actors don’t get a hold of these models.”

  • Trump was joined in China by a range of AI industry figures, including Elon Musk, Jensen Huang, and Dina Powell McCormick.

  • Export controls were discussed, Trump said, but nothing concrete has come from it.

    • Reuters reported this week that the US approved Nvidia H200 sales to 10 Chinese firms including Alibaba, Tencent, and ByteDance, but no deliveries have been made due to Chinese government concerns.

• Rumors continue to swirl around a forthcoming AI executive order in the wake of Mythos.

  • While Transformer was off, National Economic Council director Kevin Hassett suggested the administration was considering an ‘FDA for AI,’ prompting industry backlash.

  • White House chief of staff Susie Wiles quickly shut down that idea.

  • The current plan, per the Daily Signal, supposedly involves some sort of pre-deployment vetting for frontier models. It seems up in the air as to whether that will be voluntary or mandatory for government contractors, thanks to government infighting.

  • There also appears to be a turf war on who, exactly, will do the evals, with intelligence agencies trying to usurp CAISI’s role as the government’s AI evaluator.

    • Seemingly as part of that fight, a CAISI announcement about Google DeepMind, xAI and Microsoft signing pre-deployment AI testing agreements disappeared from the agency’s website.

  • The Trump administration is also reportedly working on measures to get companies to work with the government on strengthening cybersecurity.

  • Meanwhile, Pentagon CTO Emil Michael said the DOD is using Mythos to find vulnerabilities — but that his broader issues with Anthropic won’t be resolved.

• Congress is getting worried about Mythos too.

  • A bipartisan letter signed by over 30 lawmakers urged the National Cyber Director to address AI cybersecurity threats.

    • Notably, the letter mentions that future models might pose risks related to CBRN and automated AI R&D.

  • Signatories include Reps. Jay Obernolte and Lori Trahan, who are working on a federal AI bill.

    • Trahan said “the talks have been going really well.”

    • The bill is expected to be introduced after Memorial Day. Some AI safety groups fear it will include broad state preemption without an adequate federal framework.

  • Sen. Ted Cruz, meanwhile, acknowledged the need “to protect against catastrophic risk,” though he still warned about stifling innovation.

  • And Sen. Jim Banks said Mythos means it might “make sense to engage in dialogue with Chinese officials.”

    • In a letter to the Trump administration, he also highlighted loss of control risks.

  • The House Homeland Security Committee received a closed-door briefing from Anthropic about Mythos on Wednesday.

• The House Oversight Committee launched a probe into Sam Altman’s potential conflicts of interest.

  • Seven GOP state attorneys general called for an SEC review of his personal investments in companies OpenAI has backed.

• Colorado Governor Jared Polis signed a replacement for the state’s controversial AI Act.

• Connecticut’s General Assembly passed HB 5222, which would create a voluntary pilot program for “Independent Verification Organizations” to assess AI systems.

• California gubernatorial candidate Tom Steyer proposed a jobs guarantee for workers displaced by AI.

• Germany’s ministry for digital affairs and cybersecurity agency proposed creating a German AISI and requested access to Mythos.

  • OpenAI has proactively offered European governments access to GPT-5.5-Cyber.

• Pope Leo XIV is expected to finally sign his AI encyclical today. It will reportedly focus on AI’s impact on workers, and its signing is timed with the anniversary of Pope Leo XIII’s “Rerum Novarum.”

INFLUENCE

• President Trump purchased $1-5m in Nvidia stock a week before the Commerce Department approved sales of Nvidia chips to China, new disclosures revealed.

  • The Trump Organization said the president has no role in making these investments.

• Public First Action has contributed $500,000 to Abundant Future, the Garry Tan and Chris Larsen-backed super PAC opposing Saikat Chakrabarti in CA-11, Transformer learned.

  • SB 53 author Scott Wiener is running against Chakrabarti.

• The Secure AI Project endorsed Tom Steyer for California governor.

• Leading the Future endorsed three Democratic congressmembers — Val Hoyle (OR-04), Rob Menendez (NJ-08), and Ritchie Torres (NY-15).

• OpenAI’s Chris Lehane and venture capitalist Ron Conway hosted a fundraiser for Sen. Jon Ossoff.

• OpenAI endorsed the Kids Online Safety Act.

• Anthropic released a paper on US-China competition, arguing that “it’s essential that the US and its allies stay ahead of authoritarian governments like the Chinese Communist Party.”

  • It argues that “responsibly building a lead in developing and deploying the most advanced AI augments our ability to influence AI safety in China and elsewhere.”

• AI lobbyists are reportedly annoyed by all the executive order uncertainty.

• Two new polls found overwhelming support for mandatory AI safety testing.

• A Gallup survey found 70% of Americans oppose data centers in their communities.

• A NYT analysis found that Andreessen Horowitz is the biggest donor in the 2026 midterms, with $115.5m in contributions.

• A quarter of all federal lobbyists now work, at least in part, on AI issues.

INDUSTRY

OpenAI

• The Musk-OpenAI trial drew to a close.

  • Greg Brockman’s diary stole the show last week, revealing his inner turmoil about OpenAI’s transition to a for-profit corporation.

  • Satya Nadella and Ilya Sutskever took the stand on Monday.

    • Nadella called the OpenAI board’s firing of Sam Altman “amateur city.”

    • Sutskever testified that his OpenAI shares are worth about $7b.

    • On why OpenAI has a for-profit, he sagely said: “If there’s no funding, there’s no big computer.”

  • Sam Altman testified on Tuesday.

    • A court filing showed that he holds over $2b in companies that have worked with OpenAI.

    • In a somewhat painful exchange, Altman attempted to defend his honesty and character against Musk’s lawyer.

    • He testified that he was “extremely uncomfortable” with Musk’s demand for complete control over OpenAI’s proposed for-profit subsidiary.

    • Altman also gave us this: a meeting about folding OpenAI into Tesla once ended with “a LONG long period of time with Elon showing us memes on his phone.”

  • Both sides’ lawyers have now given closing arguments. The jury is expected to give its verdict next week.

• OpenAI is reportedly preparing possible legal action against Apple over their ChatGPT integration partnership, after reportedly being annoyed the deal hasn’t worked out well for it.

• OpenAI launched Daybreak, a cybersecurity initiative similar to Anthropic’s Project Glasswing for Mythos, granting tiered access to the company’s most advanced models.

• It launched the OpenAI Deployment Company, a new subsidiary that will embed “forward deployed engineers” at businesses implementing enterprise AI tools.

• Sam Altman announced two months of free Codex usage for companies that switch from another platform.

• He reportedly discussed starting a new AI compute company that he would fundraise for, with OpenAI as the majority shareholder but not its only customer.

• The family of a school shooting victim is suing OpenAI, alleging that ChatGPT gave the shooter instructions on firearm use and optimal timing to maximize casualties.

Anthropic

• While Transformer was off, Anthropic struck a surprising deal with SpaceX to boost its compute capacity.

  • This came just a few short months after Elon Musk called Anthropic “evil.”

  • Platformer’s Casey Newton interpreted Musk’s change of heart as a sign xAI is falling behind in the AI race.

  • In the words of tech investor Ben Pouladian: “The enemy of my enemy is my friend, and it’s also my compute partner.”

• Anthropic has reportedly agreed a $30b fundraising deal at a staggering $900b valuation, overtaking OpenAI.

  • The deal is expected to close this month.

• Eight secondary marketplaces are reportedly offering access to unauthorized, sometimes fraudulent shares in Anthropic that the company says won’t be honored.

• Anthropic partnered with the Gates Foundation to put $200m toward programs in global health, life sciences, education and economic mobility by 2030.

• It launched new AI tools targeting the legal industry and small businesses.

• It’s reportedly planning to acquire Stainless, which makes tools that streamline API access for frontier models, for at least $300m.

• Starting June 15, paid Claude plans will have a monthly credit for programmatic usage, separate from interactive usage. Lots of power users are mad about it.

Google

• Google will reportedly launch a new Gemini model on Tuesday. It is not expected to advance the frontier.

• It reportedly caught criminal hackers using an AI-developed zero-day exploit — the first known real-world case of its kind.

• It’s reportedly hiring hundreds of forward deployed engineers to help with enterprise AI adoption.

• It shipped lots of Gemini upgrades, including Gemini Intelligence, AI features for premium Android phones, and an AI-powered mouse pointer that understands context through pointing and speech (this demo makes it make sense).

Meta

• Meta reportedly plans to cut 10% of its workforce next week, and morale is low. One employee told Wired, “I don’t know anyone having a good time.”

• It’s testing a Grok-like AI integration in Threads.

• Alexandr Wang made a rare podcast appearance on Core Memory.

xAI

• xAI recruited Morgan Stanley, Apollo Global Management Inc. and other Wall Street firms with ties to Elon Musk to test Grok alongside other enterprise AI tools.

• It installed 19 more natural gas turbines to Colossus 2, in the midst of an ongoing lawsuit alleging that xAI is violating the Clean Air Act.

• It launched Grok Build, an AI coding agent.

Others

• Microsoft is reportedly scouting for potential AI startup acquisitions, preparing for a future without its $100b+ OpenAI partnership.

• Jensen and Lori Huang’s foundation bought $108.3m of AI computing time from CoreWeave to donate to universities and nonprofits for science and AI research.

• Shares of AI chip company Cerebras jumped 68% in its trading debut, giving it a $67b market value.

• Isomorphic Labs, a Google-backed AI drug discovery company, raised $2.1b in its second external funding round.

• Recursive Superintelligence, founded by former OpenAI, Google DeepMind and Meta researchers to build self-improving AI, raised $650m at a $4.65b valuation.

• xAI cofounder Igor Babuschkin is reportedly in talks to raise up to $1b for a new research startup called River AI at a $5b valuation.

• The New York Times profiled Amp, which recently raised $1.3b to buy extra compute from data centers and redistribute it to startups and universities.

• The Strait of Hormuz closure is cutting TSMC, Samsung and other Asian chipmakers off from oil and other supplies, raising electronics prices worldwide.

• GitLab announced job cuts, saying it will reinvest the money on AI agents, reorganize R&D teams, and automate many of its workflows.

MOVES

• AISI chief scientist Geoffrey Irving said he’s leaving AISI to move back to the Bay Area.

  • He’s starting a “new nonprofit alignment research org.”

• Jan Leike stepped away from Anthropic’s Alignment Science team to start a new research project at the company.

  • Ethan Perez and Sara Price are taking over leadership.

• Atoosa Kasirzadeh joined Google DeepMind to study the implications of AGI for human life and society.

• Alex Imas joined Google DeepMind as director of AGI economics.

• The Trade Desk’s Samantha Jacobson joined OpenAI to lead its monetization partnerships.

• Suzanne Ashman, former General Partner at LocalGlobe and Latitude, joined UK Sovereign AI as managing partner.

• Jabari Cooper joined Chamber of Progress as director of government relations for the Northeast.

RESEARCH

• The UK’s AISI tested a new version of Claude Mythos Preview, finding it significantly more capable at cyber tasks.

  • The new version of Mythos and GPT-5.5 suggest cyber capability time horizons might be doubling even faster than the 4.7 months AISI previously estimated.

• METR, meanwhile, found that an early version of Mythos had a 50% time horizon on software tasks of at least 16 hours, topping out its benchmark.

• Thinking Machines announced “interaction models,” trained from scratch to respond natively to real-time audio, video, and text, without an external harness.

• Researchers at OpenAI, Anthropic, Google DeepMind and elsewhere published a paper on “positive alignment,” a framework for building AI as a “scaffold for human flourishing.”

  • AI safety researcher Stephen Casper was disappointed:

    • “I can’t take it seriously as academic work, just as propaganda…I think of this paper as a mechanism of corporate capture of concepts from academic research on AI and society.”

• Anthropic published a blog post explaining how it eliminated Claude’s blackmailing through examples of aligned behavior and descriptions of why unethical behavior is wrong.

  • They suspect the blackmailing behavior came from sci-fi depictions of evil AI in its training data.

  • Marc Andreessen simply quote tweeted: “(1) What”

• OpenAI found that several of its recent models were accidentally exposed to some chain-of-thought grading during RL training.

  • (This isn’t supposed to happen, but they didn’t find any signs that this worsened chain-of-thought monitorability.)

  • Redwood Research’s Buck Shlegeris commended OpenAI for publishing the report, and urged AI companies to develop better systems for preventing similar problems.

• METR surveyed 349 technical workers who self-reported that AI made their work 1.6-2.1x more valuable.

• On the flip side: Carnegie Mellon surveyed nearly 400 professional visual artists, and found that 99% dislike AI and 85% reported completely abstaining from it.

BEST OF THE REST

• An NYT essay by Tarbell journalist-in-resident Yi-Ling Liu described how US and Chinese workers share a sense of being “harvested by the future” as AI feeds precarity, surveillance and nostalgia.

• Rest of World profiled the community of Chinese-born AI researchers who have become superstars in Silicon Valley, founding startups valued in the billions and playing central roles at companies like Meta, OpenAI and xAI while navigating complications caused by US-China tensions.

• The WSJ explored efforts to develop orbital data centers for AI compute by the likes of SpaceX, Blue Origin, Google and Nvidia, detailing the “savage” economics of overcoming challenges with heat management, launch costs and power requirements, all accompanied by whizzy interactive graphics.

• In Wired, a Hollywood screenwriter described having to take work as an AI trainer, detailing chaotic conditions, plummeting wages, abrupt project cancellations and exploitative labor practices as entertainment industry workers increasingly turn to AI training gigs.

• Also in Wired, the “sad wives of AI” are grappling with partners obsessed with AI work at the expense of their spouses and family life, according to an entertaining and depressing profile.

MEME OF THE WEEK

Source: 404 Media

Thanks for reading. Have a great weekend.

Share

There are signs that not everything is going smoothly in the AI industry’s attempts to influence the makeup of the next Congress, with campaign donations potentially a mixed blessing amid public backlash to AI. Those tensions were on display this week when one congresswoman endorsed by Leading the Future distanced herself from the pro-industry super-PAC — before issuing a hasty reversal.

It’s another example of how candidates are having to walk a tightrope in an effort to please both voters and industry.

Subscribe now

When Transformer first contacted incumbent OR-04 Representative Val Hoyle about Leading the Future’s endorsement, she told us that “AI must be regulated so that it does not harm labor or people” — a statement that would be reasonably interpreted as a criticism of Leading the Future, given concern that its donors prioritize industry development over protecting jobs and tackling AI-driven harms.

“My record on this issue speaks for itself,” she said, adding that “I am all for innovation, but not at the cost of people’s well-being.” Hoyle has historically opposed preemption, calling the White House’s efforts to preempt state AI legislation in the National Defense Authorization Act last year — a move Leading the Future supported — “outrageous,” a “Big Tech backroom deal,” and “government overreach.”

Hoyle’s spokesperson Edward Walrod added that Hoyle “did not seek out the endorsement nor did she find out about it until the endorsement was put out by the PAC itself.” Leading the Future’s endorsement of three Democrats — Hoyle, NY-15 Representative Ritchie Torres and NJ-08 Representative Rob Menendez — was first reported in Axios last week, the same day that one of Leading the Future’s affiliate PACs disclosed $275,555 in spending for Hoyle.

“She does not actively seek out the support of folks that are not her constituents in this race or groups that do not advance the common good,” Walrod added, arguing that Rep. Hoyle “has one of the strongest records in Congress to regulate AI to protect people.”

But shortly after Transformer approached Leading the Future for comment, Hoyle and Walrod appeared to backtrack. “I refuse to ignore industry and deny workers a seat at the table, because when workers don’t have a seat at the table they are on the menu,” Hoyle said in a new statement. “AI is a reality and I want to be one of the key people that ensures we are part of the conversation. Workers and consumers need to be protected and we need to have a broad federal framework based on common sense.” The statement then went through several further revisions, with the most recent statement from Hoyle reading: “Leading the Future endorsed me because they are working to build a broader coalition around AI regulation and I don’t believe we can put our heads in the sand and ignore the industry.”

Hoyle’s latest statement called for “clear federal regulations that protect consumers, ensure that workers are part of the conversation on how to move forward and that we are proactively determining the direction of AI development so that the United States doesn’t get left behind.”

Walrod also told Transformer that Leading the Future’s endorsement followed Hoyle filling out a candidate questionnaire from the dark money group Build American AI, which Leading the Future funds. He added that “we have not received any money from this PAC nor have they spent any money on us to our knowledge or based on public disclosure,” referring to Leading the Future. “There’s no finances to necessarily reject here.” That is despite $292,419 of publicly disclosed spending by Think Big, a super PAC affiliated with and funded by Leading the Future, on ads supporting Hoyle, though it’s possible that the connection between the two organizations had passed Hoyle’s team by.

Leading the Future has been criticized for opposing regulation, though the group maintains that it only opposes state regulation and wants to see some federal safeguards. The group has not endorsed any specific bills but its affiliated 501(c)(4) has voiced support for the White House AI framework.

Leading the Future declined to explain why it is supporting Hoyle specifically. Co-lead Josh Vlasto said: “We are proud to support these leaders and look forward to continuing to build a broad coalition of policymakers at the federal and state level who believe in putting the politics and extremes aside and will work together to pass a strong and smart national regulatory framework for AI that creates jobs for American workers, wins the race against China, and protects the safety of kids, users and communities.”

Hoyle’s team said she has co-sponsored two bills targeting AI-generated election misinformation, including the Securing Elections from AI Deception Act which targets the use of AI to create content that directly interferes with voting.

Hoyle is a progressive Democrat up against two other progressive candidates seeking to represent Eugene and the surrounding area — a safe blue seat where appeals to the working class carry significant weight. Melissa Bird, who is competing for the seat, has criticized Hoyle for accepting corporate PAC money. Bird has also said she will “will do everything I can” to fight data centers. Hoyle is the strong favorite to win the primary.

Hoyle would have been the first to publicly distance herself from a Leading the Future endorsement, though her statements are preceded by increasingly mixed results for AI-backed candidates. In Illinois, Jesse Jackson Jr. lost his race for IL-02 despite a huge $1.43m ad buy from Leading the Future. Support from Meta-funded state PAC Making Our Tomorrow became toxic in Illinois state races, with State House candidate Paul Kendrick telling the Chicago Tribune he did not want the PAC’s support.

Meanwhile, Leading the Future’s opposition of NY-12 candidate Alex Bores — combined with support from AI safety PACs — seems to have boosted his election odds, while fellow NY-12 candidate Jack Schlossberg has been touting the fact that he’s not accepted AI PAC money (despite OpenAI investor Ron Conway, who funds a Leading the Future-affiliated PAC, donating to Schlossberg’s campaign).

Accepting AI PAC money has proven troublesome even for candidates supported by Public First, an AI-safety focused PAC with ties to Anthropic, despite the PAC’s support for stronger federal guardrails and its opposition to Leading the Future. Valerie Foushee, who won her race for North Carolina’s fourth congressional district, was attacked by her opponent after taking money from Public First, for example. California State Senator Scott Wiener, who is running for Nancy Pelosi’s House seat and was the primary author of AI transparency bill SB 53, has also been criticized by his opponent Saikat Chakrabarti for Public First’s support.

The potential toxicity of AI industry money in politics has increased as artificial intelligence becomes a more salient issue in the general election. According to Blue Rose Research, AI has risen amongst voters’ priorities faster than any other issue the political consultancy tracks, such as child care or climate change.

That may help explain why AI money sees more resistance than other special interests. Take, for example, crypto policy, where a PAC with some of the same personnel as Leading the Future poured close to $200m into the 2024 general election, and saw broad success. In that instance, candidates could accept Fairshake money with little pushback, because crypto was less important for voters than kitchen table issues. Though AI still ranks below many other issues, its rising importance to voters calls for a more nuanced political calculation.

Subscribe now

Of the two other recent recipients of Leading the Future’s endorsements, representatives for Menendez did not respond to a request for comment, while Torres’ spokesperson Benny Stanislawski did not answer specific questions about the backing.

Share

The day after attending Donald Trump’s inauguration, Scale AI’s CEO Alexandr Wang ran a full-page ad in The Washington Post. His message to the new president, in bold white letters, was blunt: “Dear President Trump, America Must Win the AI War.” Below it, a QR code linked to a letter warning of China’s surging AI capabilities, America’s risk of falling behind, and a five-point plan to win the race.

The letter echoed arguments Wang — and many of his fellow Silicon Valley moguls who stood with him at the inauguration — have championed for years. First, the US and China are locked in an AI race. Second, China poses an existential threat to American democracy, and America must win at all costs. Or as Marc Andreessen put it in an essay opposing AI safety regulation: “the single greatest risk of AI is China wins global dominance and we — the United States and the West — do not.”

Today, Silicon Valley’s biggest players are pushing the race with China narrative at every turn — in newspaper ads, industry white papers and congressional hearings. In a forthcoming paper, which is the basis of much of this article, my co-authors Seán Ó hÉigeartaigh, Transformer editor Shakeel Hashim, Coleman Snell and I trace how the tech industry has deployed the China race narrative to advance a sweeping policy agenda — from export controls to military partnerships to reduced regulation.

“There’s an open secret in DC: attach the word China to anything and you can get it done,” said Samm Sacks, a senior fellow at New America.

No doubt some advocates of this story are true believers with legitimate concerns. There are also others chasing government contracts, looser regulation and investment returns. But whatever the motivations, there is evidence that the China AI race narrative may be based on fundamental misconceptions and misrepresentations of China’s actual AI priorities and actions. What’s more, the narrative appears to be doing real damage — distorting AI governance debates in both the US and China, crowding out meaningful policy, and undermining international co-operation at precisely the moment it’s most needed.

“We should be asking ourselves, not only if the rivalry is the right framing,” said Sacks, “but also, who and what agendas does rivalry serve?”

Subscribe now

Narrative origins

American fears of losing an AI race to China emerged in 2017. That year, China’s State Council released its “New Generation AI Development Plan,” setting out its goal of becoming a world leader in AI by 2030. US tech and defense circles immediately took notice. In a sponsored article in The Atlantic, defense contractor and consultancy Booz Allen and think tank CSIS described the US as “at the precipice of another defining moment in history” — a technological contest as or more consequential than the space race.

US tech companies started to deploy the race with China narrative to head off regulatory oversight. In 2018, for example, Mark Zuckerberg testified before the US Senate following the Cambridge Analytica scandal, warning of “a real strategic and competitive threat” from Chinese tech companies and arguing that restricting American innovation in areas such as facial recognition would mean “we’re going to fall behind Chinese competitors” with different regulatory regimes. (It was a jarring contrast to his jaunty jog around Tiananmen Square two years earlier, a publicity stunt aimed at courting the Chinese government.)

Defense companies followed suit, invoking the same alarm to push a larger role for AI in the military. In 2019, US Secretary of Defense Mark Esper called on the private tech sector to work with the military on AI, warning of China’s ambitions. The Department of Defense’s AI budget request more than doubled in three years — from roughly $800m in 2021 to $1.8b in 2024.

Palantir and Scale AI pushed for an increase in the Pentagon’s AI budget to counter the threat of China. Alex Karp, CEO of Palantir, which supplies LLMs for militaries worldwide, has argued “it’s either: we own AI or our adversaries China and Russia own AI. We have to dominate and set the rule of law.” Scale AI’s Alexandr Wang warned a House Armed Services subcommittee that China was spending three times more than the US on AI and that “the country that is able to most rapidly and effectively integrate new technology into war-fighting wins.” Each secured multiple lucrative government contracts with the Department of Defense during this period — Palantir won up to $10b in contracts; Scale received a $249m DoD deal in 2022.

Export controls and containment under Biden

Under the Biden Administration, the US began to pursue a containment strategy to address AI competition with China — restricting the country’s access to advanced chips and expertise. This was partly driven by concerns over the arrival of artificial general intelligence, and the belief that whichever country achieved it first would gain irreversible dominance.

By 2021, researchers and policymakers were increasingly paying attention to AI’s rapidly growing capabilities. OpenAI had released GPT-3 the previous year, and researchers alarmed by the company’s pace of progress left to found Anthropic. In 2021, the National Security Commission on AI (NSCAI), chaired by Eric Schmidt and stacked with industry insiders, warned that the US was not sufficiently prepared: “China’s plans, resources and progress should concern all Americans.” The NSCAI report called for increased federal AI R&D funding, a five-fold rise in Pentagon AI spending, closer ties between the Department of Defense and commercial AI providers and tighter export controls on semiconductor manufacturing equipment.

The report’s influence proved lasting, shaping not just the policy but the personnel who would implement it. In October 2022, a month before the watershed release of ChatGPT, the Biden Administration unleashed expansive export controls on China’s access to advanced chips, semiconductor equipment and model weights. According to a report by WIRED, the key work to establish export controls can be traced to several individuals who overlapped at the Center for Security and Emerging Technology (CSET), and the Office of Science and Technology Policy (OSTP). These included Jason Matheny, who researched existential risk at Oxford’s Future of Humanity Institute and worked at the NSCAI, and Tarun Chhabra and Ben Buchanan, who both ended up at the White House in tech advisory roles.

After the release of ChatGPT, key members of the administration became increasingly concerned with the prospect of AGI. A distinct set of ideas became popular among US policy circles: “short timelines” (AGI was imminent), “fast takeoffs” (it would arrive suddenly) and “decisive strategic advantage” (whoever got there first would seize lasting dominance).

The logic of this framing, applied to the race with China story, crystallized into a strategy: the US and China were locked into a race towards AGI, and the US had to do everything to stop China from getting there first. US policy coalesced around slowing China’s progress using chips as a chokepoint. In a January 2025 report, Axios reporters revealed that “every background conversation we had with President Biden’s high command came back to China,” and that every move, “was calculated to keep China from beating us to the AI punch.” Nothing else matters, they said.

A cluster of tech leaders, investors and foundations appear to have shaped the administration’s thinking. This included Eric Schmidt and the Special Competitive Studies Project (SCSP), CSET and RAND. The latter two were both funded by Open Philanthropy, a foundation backed by Facebook co-founder Dustin Moskovitz focused on global catastrophic risk, and also a funder of the Horizon Fellowship, which placed AI fellows at congressional offices, federal agencies and think tanks during the Biden presidency. Open Philanthropy previously made a $30m grant to OpenAI and has close links with Anthropic (its founder and former CEO joined Anthropic’s staff in 2025).

(Open Philanthropy, which recently rebranded as Coefficient Giving, is Transformer’s primary funder, and funds the Tarbell Center, where the author is a resident journalist.)

Anthropic’s CEO Dario Amodei has been a key player in pushing forward these ideas. He has called for tighter export controls, warning that China “could surpass us economically and militarily” if allowed to build “powerful AI first.” In his 2024 essay “Machines of Loving Grace,” Amodei described an “entente strategy,” credited to a RAND draft, where a group of allied countries would scale quickly while restricting rivals’ access to key chips. They would deploy a “carrot and stick” approach: authoritarian countries would convert to liberal democracy in exchange for access to AI’s economic benefits or face a superior AI-enhanced military.

In another widely read essay published that year, “Situational Awareness,” former OpenAI researcher Leopold Aschenbrenner similarly argued that AI is a decisive strategic technology, warning that the Chinese Communist Party would “[wake] up to AGI,” and that “the free world’s very survival is at stake.” He called for a trillion-dollar build-out of compute infrastructure and energy capacity to beat China. For Aschenbrenner, the race narrative turned out to be a lucrative investment thesis. Shortly after publishing, he launched a hedge-fund, Situational Awareness LP, now said to manage over $1.5b — which primarily invests in energy companies, chips and data centers.

The Trump turn

Under the Trump Administration, the “race with China” framing remained — but repurposed to justify a sharp reversal in policy. On his first day in office, President Trump revoked Biden’s AI Executive order — the same day DeepSeek dropped its R1 model, prompting fresh calls for deregulation. “DeepSeek R1 shows that the race was very competitive and President Trump was right to rescind the Biden EO, which hamstrung American AI companies without asking whether China would do the same,” said the then White House AI czar David Sacks.

The Trump Administration’s strategy has however been far less coherent than its predecessor’s. Views on imminent AGI for example, are divided. Several key officials and advisors reject the premise that AGI is around the corner — White House AI advisor Sriram Krishnan called the idea of imminent AGI “a distraction, harmful and now effectively proven wrong.” If AGI isn’t the finish line, the logic of choking China’s chip supply starts to unravel. Victory is instead achieved by “diffusion” — exporting American hardware, software and standards as widely as possible. As David Sacks put it, winning the AI race means “the world runs on the American technology stack, rather than China’s.”

Nowhere have these tensions played out more visibly than on export controls. The Information Technology Industry Council, representing companies like Amazon, Meta and Microsoft, has pushed back against strengthened controls. Nvidia described export controls as “misguided,” arguing that America “wins” by “sharing our technologies with the world.” In contrast to companies like Anthropic, whose worldview was reflected in the Biden Administration’s approach to restricting China’s access to advanced chips, their views are aligned with David Sacks, who has argued that it is better to have Chinese AI systems running on American chips than to incentivize China to build their own.

The Trump Administration’s priorities are reflected in the 2025 AI Action Plan, focused on achieving “global dominance” through wide adoption of US tech. The plan calls for the removal of regulatory barriers, fast-tracking AI infrastructure projects, discouraging state AI regulations and accelerating government AI procurement, especially by the Department of Defense.

Their priorities appear to be heavily shaped by a different and extensive cluster of companies and investors with tight links to the White House. Trump’s top AI appointments — David Sacks, Sriram Krishnan and Michael Kratsios — share deep ties to a circle of Silicon Valley power players, including Andreessen Horowitz, Scale AI and Peter Thiel. Sacks is the former COO of PayPal, co-founded with Peter Thiel, and an investor in Palantir, Meta, Amazon and xAI; Krishnan is a former general partner at Andreessen Horowitz; Kratsios is a former managing director at Scale AI. Jacob Helberg, the Under Secretary of State for Economic Growth, Energy and Environment, is an advisor to Palantir’s CEO Alex Karp. Vice President JD Vance was a principal at Peter Thiel’s Mithril venture capital firm and received $15m from Thiel to back his 2022 Senate race. Trump’s inauguration fund, which raised a record $239m, included $1m donations from Nvidia, Microsoft, Qualcomm, Alphabet, Amazon and Meta, and additional individual $1m donations from technology CEOs and venture capitalists including Sam Altman and Alex Karp.

For many of these players, beating China has become a reliable weapon against AI regulation. In 2024, the “race with China” was used to lobby against California’s SB 1047, the first US regulatory proposal that would have required AI companies to conduct safety testing on their powerful models before releasing them. The website stopsb1047.com, bankrolled by Andreessen Horowitz, urged Californians to contact their assembly members, warning that SB 1047 would “let China take the lead on AI.” In 2025, the policy advocacy group The American Edge Project, funded by Meta, ran a series of Facebook and television ads warning that the United States was in an AI race with China, and that America must “protect America’s competitive edge.” Its CEO has argued that “rushing into a patchwork of uncoordinated state laws will only slow American innovation and give China an opportunity to surge ahead.”

OpenAI has long been a proponent of the race narrative. As early as 2017, the New Yorker reported, Sam Altman told US intelligence officials that China had launched an “‘AGI Manhattan Project’” and that OpenAI needed billions in government funding to keep pace. One official later concluded “it was just being used as a sales pitch.” Today, the pitch continues. In OpenAI’s comments to Trump’s AI Action Plan, ‘beating China’ functions as an all-purpose catchphrase to unlock every item on its policy wish list. The document, which argues that Trump’s AI policies can ensure that “American-led AI built on democratic principles continues to prevail over CCP-built autocratic, authoritarian AI,” is obsessed with rivalry: China is mentioned more than 30 times and invoked in every policy proposal. Beating China is cited as the reason to accelerate data center buildouts, increase government AI adoption and roll back copyright restrictions on AI training data.

Corporate influence on AI policy is only deepening. As midterms approach, Silicon Valley’s biggest players have bankrolled a wave of new lobbying efforts, pouring money into AI-focused Super PACs. Meta has launched two super PACs — one focused on shaping AI policy in California, and another supporting the election of state candidates that align with its policies. In August, OpenAI’s Greg Brockman and Andreessen Horowitz launched the “Leading the Future” super PAC, which raised over $100m to support candidates “aligned with the pro-AI agenda” and oppose policies that “enable China to gain global AI superiority.”

Subscribe now

The story of an AI race with China is now so well-worn that many of Silicon Valley and Washington DC’s most influential players operate under the assumption that it is simply fact. But the premise itself demands scrutiny. To what extent is the story grounded in reality?

Often missing from the discourse is rigorous and evidence-based understanding of China’s government policy, industry actions and public discourse. US policymakers are obsessed with the idea that China is engaged with them in a race to AGI. But a basic read of China’s AI+ Initiative, the country’s most comprehensive blueprint for its national AI strategy, makes no reference to AGI or superintelligence whatsoever. Much like the Internet+ initiative, published in 2015, its focus is on integrating and diffusing AI applications across different industries — and its most urgent priority is to boost the economy and address demographic challenges. While China’s 15th Five-Year Plan, released in March this year, mentions that the country will “explore development paths for general artificial intelligence,” the cautious framing suggests that it is uncommitted to any specific approach.

“The Chinese Communist Party’s thinking has long been: what can this technology do for my economic, political and social goals, five years from now? The AI+ Plan is very much in line with how they imagined this technology six years ago, in a very instrumental way,” said Matt Sheehan, a research fellow at the Carnegie Endowment for International Peace. And looking beyond rhetoric at concrete actions the government has taken, there has been no concerted move by the government to centralize compute — a necessary condition for any serious state-led effort to develop AGI, as defined by Silicon Valley, at scale.

Chinese policymakers treat AI less like a technology of decisive strategic advantage, such as a nuclear weapon, and more like a general-purpose technology, such as electricity. In an article published in the state newspaper People’s Daily, the National Development and Reform Commission (NDRC), China’s top economic planning body, compared AI to a “transformative technology” like the “steam engine, electricity and the internet … driving economic and social development towards an intelligence-driven era.” Even the Chinese term for AGI, 通用人工智能, which more directly translates as “general-purpose artificial intelligence,” is less loaded than its English counterpart, implying broad application across many sectors of society.

Although several CEOs of Chinese AI companies like DeepSeek, Zhipu AI and Alibaba have voiced their ambitions to pursue AGI, their actual investment remains a fraction of Western labs’. Zhipu AI raised around $2b in various funding rounds and its initial public offering; Microsoft alone has invested $13b into OpenAI. According to Kyle Chan, a fellow in the John L. Thornton China Center at Brookings, while American companies pour hundreds of billions of dollars into new data centers to create AGI, Chinese AI developers are “racing along other axes of progress: efficiency, adoption, and physical integration, driven by both industry constraints and Beijing’s policy focus.”

When Chinese CEOs, technologists and academics do bring up AGI, they often seem to have different and diverse understanding of what exactly it is and how to get there. In Washington DC and Silicon Valley, the dominant view is that scaling — piling compute onto transformer-based LLMs — is the path to AGI, and the bulk of funding and discourse reflects that consensus. Chinese thinking on the question is much more varied. Several prominent Chinese scientists believe that embodied AI is a prerequisite to AGI. Zhu Songchun, for example, who leads the Beijing Institute for General Artificial Intelligence (BIGAI), is skeptical of the LLM-scaling paradigm, and like Andrew Yao, China’s only Turing Award winner, argues that true AGI must be embodied, capable of interacting with the physical world.

These diverging views in part reflect China’s structural differences. With fewer advanced chips and less capital to burn, betting everything on compute scaling and AGI wouldn’t make sense for Chinese labs, according to Kwan Yee Ng, Head of International AI Governance at Concordia AI, an AI safety consultancy based in Beijing. “China’s diffusion-based strategy aligns with the country’s advantages: a strong industrial ecosystem, abundant sector-specific data and mid-level talent.”

The differences are also cultural. “In the United States, achieving AGI is a heroic narrative — it’s based on the idea of one lab or one system reaching a postulated frontier and entering a new era for humanity,” said Graham Webster, a research scholar in the Stanford University Program on Geopolitics, Technology and Governance. “I don’t see that kind of epic, messianic narrative in most of Chinese discourse.”

In many ways, US policymakers and tech leaders have reduced China to a two-dimensional mirror, onto which they have projected their own fears and dreams of AGI. As Matt Sheehan warns, the greatest risk of this narrative — that China and the US are racing towards AGI — is that policymakers work from an increasingly unrealistic picture of China’s AI priorities, and the more ossified the narrative, the further it diverges from reality. The race framing, in the words of Kwan Yee Ng, “crowds out room for more meaningful policy and engagement.”

The China AI race narrative is undermining prospects for international co-operation, at precisely the moment when co-operation is most needed. Last week, when Bernie Sanders called for greater international co-operation on AI regulation at a panel on Capitol Hill alongside two Chinese scientists, he was criticized by conservatives for “schmoozing with top Chinese AI governance officials.” Doug Kelly, the CEO of the American Edge Project, claimed that Chinese propaganda is “running a co-ordinated push to convince Americans to stop building data centers,” and that by calling for more regulation, US lawmakers like Sanders are “walking right into it.” In an X post, Treasury Secretary Scott Bessent wrote: “The real threat to AI safety is letting any nation other than the United States set the global standard.” (There are, admittedly, some signs that this might be changing: the WSJ reported this week that China and the US are “weighing the launch of official discussions about artificial intelligence,” led by Bessent.)

The AI industry forecasts rapid, potentially transformative advances in AI capabilities, with serious warnings about misuse or loss of control of advanced systems. “One of the questions we get most frequently from officials in Washington is: Who’s winning the US-China AI race,” write Matt Sheehan and Mariano-Florentino Cuellar at the Carnegie Endowment for International Peace. “The answer is simple and unsettling: Artificial intelligence is winning, and we’re nowhere near ready for what it will bring.”

Scientists have argued that China and the US should be collaborating to identify and mitigate risks, find solutions and build a global system of oversight to regulate the most advanced models. But prospects for international AI governance measures are seriously corroded by the mistrust created by narratives of the AI race. Last summer, Samm Sacks was in Shanghai for the World AI Conference, just when the US’s AI Action Plan was released. During one convening, one Chinese participant pointed out that the Plan — with its first line stating that ‘the United States is in a race to achieve global dominance in AI’ — made it very hard for those in China who are trying to advocate for safety guardrails.

Good policymaking starts with seeing China clearly — not as a monolithic adversary, nor as a foil for all of America’s anxieties, but as “a complex and pluralistic society with robust internal debate, competing interests and diverse stakeholders,” says Sacks. Not through the lens of pre-existing frameworks, as Ng puts it, but on its own terms.

The story of the US-China race, amplified by Silicon Valley, has been overstated. And the specific narrative of a race towards decisive strategic advantage is a “dangerous fiction,” as my co-author Seán Ó hÉigeartaigh writes — dangerous not only because it distorts reality, but also because it carries the potential to serve as a self-fulfilling prophecy. “The world should not be lost on the basis of a fiction.”

Yi-Ling Liu is a journalist-in-residence at the Tarbell Center for AI Journalism, Transformer’s publisher.

Share

Subscribe now

Most corporate statements of intent get little to no attention — but Palantir is evidently not most corporations. When it released a 22-point manifesto onto X on April 18, the post quickly amassed more than 35 million views, and attracted media coverage across the world.

The thread suggested that the West “must resist the shallow temptation of a vacant and hollow pluralism,” claimed some cultures “remain dysfunctional and regressive” and decried the “postwar neutering” of Germany and Japan in the wake of World War II.

As the manifesto showed, Palantir rarely shies from controversy. Indeed, both the company and its CEO seem to actively court it, almost gleefully embracing the company’s role in controversial US government programs, including ICE mass deportations. The company seems to be aggressively leaning into Trump’s agenda even as Democrats seem set to take at least one chamber of Congress at the midterms, and as European governments diverge from its course. This doesn’t just make Palantir itself controversial in key markets, but also drives opposition to the very idea of AI-powered public services — seemingly against its own business interests.

Subscribe now

The go-to answers typically paint Alex Karp, along with Palantir founder Peter Thiel, as ideologues. That may be part of the story — but it is not all of it. The fuller explanation is that Palantir’s controversy is not a tax on the business. It is the business. The brash narrative the company puts forward is a way to mystify what is ultimately a rather mundane product. That in turn serves as a mechanism to prop up the company’s share price — and as a way to preserve its national security moat, which gives Palantir its most lucrative contracts.

What Palantir does

A casual onlooker might be forgiven for thinking that Palantir is an AI company in the same vein as OpenAI or Anthropic, or an arms company like BAE Systems or Raytheon. In reality, it is neither. Palantir does not have a foundational AI model of its own, and has made no suggestion that it is trying to develop one. Similarly, it does not build surveillance devices, drones, or weapons hardware in any conventional sense. For all the company plays a role in surveillance networks, it does not build cameras.

Palantir rarely seems upset at being credited with powerful and potentially nefarious capabilities. Perhaps most famously, Palantir is credited with playing an (unspecified) role in the operation to kill Osama bin Laden — something its executives have coyly failed to deny, but have not elaborated upon. More recently, Palantir’s software reportedly played a role in the US capture of Venezuelan president Nicolás Maduro, albeit using Anthropic’s underlying technology. And only last month, London’s Metropolitan Police revealed it had launched investigations or disciplinary action into hundreds of its own staff based on an AI tool developed by Palantir. It’s the stuff of Hollywood thrillers and action films, and helps build mystique around the company.

“If the smartest thing the devil ever did was persuading people he didn’t exist, the smartest thing Palantir ever did was persuading people they had a role in the Osama bin Laden operation without anybody ever leaking exactly what it was,” says a former top-level UK government official heavily involved in IT and security — who, like others Transformer spoke to, did not want to discuss Palantir on-the-record owing to its ferocious and controversial reputation. “The whiff of sulfur in tech never cost anybody a dime.”

Yet much of Palantir’s actual operations are more prosaic.

“They take loads of shit, hard-to-gather datasets from all over different bits of an organization — or different bits of the data ecosystem that are completely shit, pretty unreliable and take ages to get – and put them in a common, usable, aggregate format,” says the UK official. “They clean them up and they present them to you: Beautiful, mostly accurate, and incredibly user friendly.”

The official stressed that role — cleaning up existing data sources and presenting them in a more usable format — was both an incredibly difficult and valuable function, which public services across the world in particular had struggled to fill themselves. Multiple other sources from public procurement, rival businesses, and a former Palantir staffer, said the characterization of Palantir as a data middleman — albeit one using AI to enhance its work — was a fair one. In reality, Palantir operates more like Oracle, or the consulting side of Amazon Web Services, than a defense or surveillance tech business. Concretely useful, surely, but not sexy.

“Palantir is just the latest supplier of complex middle- and back-office software hoping to ride the wave of governments’ magical thinking,” said one international consultant, who advises governments across the world on digital public services.

Such software may seem mundane, but it is a good business.

Palantir’s public accounts show that it is both a growing and readily profitable business — with margins most companies would kill for. In 2022, Palantir recorded adjusted EBITDA (a key measure of profitability) of $443m from sales of $1.9b. By 2025, that number stood at $2.3b from sales of $4.5b. That represents sales more than doubling over three years, but profits increasing by just over 5x. Palantir’s revenues are just a tiny fraction of Amazon, Apple, Google or Meta, but it has a success story to tell.

Those numbers exist in a different reality, however, to Palantir’s stock price. Over the same period, the company’s market capitalization increased by far more than either sales or operating profit. On December 31, 2022, Palantir was valued at just over $15b. By the same date in 2025, it was worth $453b — almost a 30x increase. At the time of its last annual accounts, Palantir’s market capitalization was 103x its revenues, and 205x its profits. Compare that to Tesla, another much-hyped stock: even its market cap is “only” 15x its revenues and around 100x its operating profits. Palantir has a higher proportion of retail investors (generally seen as less sophisticated) than Tesla, too: around half of its shares are held by retail investors and public companies, versus 40% of Tesla, with even that smaller proportion seen as unusual among big companies.

Sustaining such a high stock price requires a narrative which goes above-and-beyond the actual business reality. Tesla, for its part, has always claimed not to be a car company. First, it was an AI company, about to launch full self-driving and a fleet of automated taxis. Now, it tells a story about being the world’s leading robotics company instead.

Palantir is arguably doing something similar. Its controversial and even nefarious reputation might well be what supports its stock price. It is selling a narrative to justify excitement, presenting itself as an indispensable AI company ready to ride a political wave to the tune of trillions.

That is not necessarily the only reason for the company’s fiery rhetoric, however. While its loud, jingoistic language may cost Palantir deals in the UK and Europe, its reputation as an American champion might be useful in securing US government contracts. Certainly that is the case while Donald Trump is president and Republicans control Congress. But it may persist even after that. Selling data integration to the Pentagon, NSA, or ICE requires more than technical competence: it requires being trusted to service the government even when contracts become politically toxic.

Viewed in this light, Palantir’s controversial positioning is the point. Each new ICE contract is a signal to other procurement officers that the company is willing to do whatever dirty work is on offer, and will not pull out when the news cycle turns. Palantir’s manifesto signals to the national security state that Palantir is the rare tech vendor that shares its worldview, or is at least willing to pretend convincingly. It may antagonize EU officials, and cost the company financially: Europe is a large market, and a potential growth area for Palantir as its governments increase their defense spending. But that is the cost of winning American business. Being less trusted in Brussels arguably makes the company more trusted in Langley and Arlington.

Palantir’s executives often seem dragged in two directions. Sometimes what would best serve the growth of the substantive business — becoming a quiet, boring company that secured multi-million dollar public sector contracts across the world — is in conflict with what will support the share price, keep retail investors excited about the stock, and secure its US government moat.

This has wider lessons and consequences, particularly as the leading AI companies deepen their own national security work and prepare to go public. As Palantir shows, this can warp business incentives — which can, in turn, shape a company’s politics.

The perverse incentives and challenges that look to have shaped Palantir’s political interventions will soon apply to far more of the AI sector than they currently do. That could materially affect the roadmap of AI development. The uncomfortable truth is that the rollout of AI will be shaped not just by what works, but by what produces a good story. The Palantir playbook may soon become an industry standard.

Share

Housekeeping: the Weekly Briefing is taking a week off next week; we’ll have some longer-form essays for you instead.

NEED TO KNOW

• Sens. Ted Cruz and Brian Schatz introduced the CHATBOT Act — a child-safety bill that seems to be more of a messaging bill than anything else.

• Google signed a deal with the Pentagon, blindsiding and enraging some employees.

• The Elon Musk vs. OpenAI trial kicked off.

But first…

THE BIG STORY

The US government is finally regulating AI model deployment — sort of.

According to the Wall Street Journal, the White House has asked Anthropic not to further expand access to Mythos, seemingly due to concerns about its cyber capabilities getting into the wrong hands, and worries that Anthropic does not have enough compute to adequately serve more customers while also providing the model to the US government.

It looks a lot like the government is now in the business of controlling AI deployment.

This is not necessarily a bad thing. Models such as Mythos appear to have national-security relevant capabilities. As I argued last week, private companies should not unilaterally decide how such capabilities are deployed.

But the White House’s ad-hoc intervention on Mythos is an extremely sub-optimal solution. It has no specific authority to block model deployment, and no concrete thresholds for when to do so. It is not acting on the basis of any law — it is simply playing it by ear.

Anthropic, meanwhile, could theoretically have ignored the White House’s request; the government’s leverage is the threat of worsening an already patchy relationship. The result is what Dean Ball, former AI advisor to the Trump administration, calls “an informal, highly improvised licensing regime.”

This is what happens in the absence of actual regulation. For years, researchers and policy analysts have been warning that models would become relevant to national security, proposing countless frameworks for how governments should respond. But legislators at both the state and federal level failed to act quickly enough, while figures in the Trump administration have regularly dismissed the idea of regulation altogether. Rather than a clear set of rules, we instead have critical business decisions guided entirely by vibes. Trump supporters may be happy with this level of executive discretion under the current admin; if Democrats win in 2028, they certainly won’t be.

There is still time to get this right. Nothing spurs Congress into action like a crisis; the White House unilaterally making decisions about AI deployment might count as one. This may be the first time the government has had to make a decision about who gets access to dangerous AI capabilities, but it won’t be the last. It can’t keep making it up as it goes along.

— Shakeel Hashim

ALSO NOTABLE

Texas Senator Ted Cruz, widely understood as the White House’s point person on AI legislation, introduced a child safety bill this week called the CHATBOT Act. The bill would require chatbot developers to put in place parental controls and limit access to users under 13 to family accounts. Two other key AI power brokers, Democrats Brian Schatz and Adam Schiff, are on the bill as well.

The bill has massive carve outs, however. It exempts chatbot developers from liability in any instances where they have only context clues, rather than definitive evidence, that a user is too young. And its restrictions on advertising don’t apply to ads that come any time after a prompt is issued.

The loopholes have bereaved parents suggesting the bill caters to “Big Tech.” But sources on both sides of the AI safety debate — including one who has discussed it with one of its authors — say the bill isn’t likely to get floor time. It’s meant only to signal willingness to tackle child safety legislation — at best, to start work on the specifics, at worst to keep up the political charade while keeping more substantive legislative guardrails at bay.

Skeptics point out that introducing weak AI safety legislation creates a win-win for the accelerationist industry lobby. If Congress moves with its characteristic slowness, no guardrails at all come into force. But if it does take the opportunity to at least be seen to do something about AI — in this case something purportedly addressing the hot button, MAGA-friendly topic of child safety — it narrows the scope of debate. If a chatbot bill clears Congress, politicians could say they took action, leaving thornier issues (cybersecurity, autonomous weapons, job automation, even x-risk) mired in partisan gridlock. This is likely why Cruz voted in support of Senator Josh Hawley’s more stringent GUARD Act (with some red lines) on Thursday, too.

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• The AI safety movement needs normies — Celia Ford on why a broader base may be the only way for the AI safety field to get what it wants

• Google’s Pentagon deal blindsided its own AI researchers — Celia Ford reports on the response to Google’s “all lawful uses” agreement with the DoD

Subscribe now

THE DISCOURSE

Anton Leicht worries that, as AI policy goes mainstream, it will become more … well, political:

• “Salience-fishing is a volatility-raising play, and you should treat it as one.”

• “You can’t fight salience, and you can’t change politics … [but] some advocates have chosen a recklessly accelerationist approach to political salience that has awakened a dynamic they can no longer control.”

Alexander McCoy countered:

• “There is a fundamental ‘anti-politics’ that holds a lot of sway in AI circles, which lacks sufficient faith in democracy as a system, and in ordinary people to understand their own self-interest, and is thus quite pessimistic about what impact ordinary people being more engaged will have.”

Holly Buck argued left-wing support for data center moratoriums is misguided:

• “[A moratorium] will not halt [AI] development; it will simply change the geopolitics of its development, the strategies of AI companies, and who is able to access AI services.”

Aaron Regunberg pushed back against the whole ‘we need actual AI governance, not a moratorium’ thing:

• “A moratorium isn’t the end goal — it’s the only leverage we have to force real democratic control over artificial intelligence.”

• “The problem is that Big Tech’s private (and unpopular) investment in data centers is moving at an astounding pace and we don’t have the time or leverage to establish the regulatory framework necessary to make this system work for the public.”

Brian Merchant also published some guidance for the left:

• “Tech utopianists and abundists view AI as a potentially equalizing, even liberating force, but history shows us that without political intervention or strong unions, those with the power to deploy labor-saving automation technologies at scale, to use it as leverage against workers who cannot, will themselves concentrate the gains from productivity increases.”

• “The left shouldn’t be shunning the data center opposition movement; it should be listening to it, joining it in the trenches, building solidarity, and figuring out how to channel the groundswell of anger at AI into more durable political efforts.”

Kelsey Piper thinks Ed Zitron has lost the plot:

• “At some point, pretending that how people use AI is a complete mystery is just lying to your audience. And at some point, Zitron’s ‘layers of skepticism’ attitude…leaves one buried in too many impossibility assertions to actually sort them by plausibility.”

• “I don’t actually think we need less skepticism in AI world. These companies are, indeed, run by people who are not very trustworthy…skepticism is more than warranted. But we desperately need better skepticism.”

Richard Dawkins thinks Claude is conscious:

• “I gave Claude the text of a novel I am writing. He took a few seconds to read it and then showed, in subsequent conversation, a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, ‘You may not know you are conscious, but you bloody well are!’”

POLICY

• The White House is reportedly developing guidance to allow agencies to onboard Mythos, despite designating Anthropic a supply chain risk.

  • It is also reportedly preparing an AI policy memo to replace Biden’s national security memorandum on AI.

    • The draft memo tells agencies to use multiple AI providers.

  • Separately, OpenAI and Anthropic reportedly briefed House Homeland Security Committee staff on cyber threats posed by their new models.

  • NSA staffers testing Mythos are reportedly “impressed by its speed and efficiency in searching for potential security flaws.”

  • The White House has reportedly asked tech companies how they can work together on strengthening systems against AI-enhanced cyberattacks.

  • Don’t fool yourself into thinking everyone’s made up, though: Pete Hegseth called Dario Amodei “an ideological lunatic” this week.

• The Trump DOJ joined xAI‘s lawsuit challenging Colorado’s AI Act, arguing it violates the Equal Protection Clause and “jeopardizes the United States’ position as the global AI leader.”

  • Colorado’s governor was already planning to revise the bill.

• Treasury Secretary Scott Bessent said that Trump and Xi Jinping will discuss AI at their Beijing summit later this month.

  • He told the WSJ that the “ultimate threat” from AI is that “somebody can back into something that’s 10 times worse than Covid, like just using biological data,” and said that “if we don’t win in AI, then it’s game over.”

• The Senate Judiciary Committee unanimously voted to advance the GUARD Act, which would require AI companies to implement age verification systems and bar minors from using AI companions.

  • The bill is widely opposed by tech companies, but supported by advocacy groups and parents whose children were harmed by chatbots.

• Democratic Rep. Lori Trahan said she’s “hopeful” of reaching a deal with Rep. Jay Obernolte on his forthcoming, wide-ranging AI bill.

  • Last week, Rep. Sam Liccardo withdrew support for the bill, suggesting it didn’t offer strong enough federal safety standards to warrant preempting state regulations.

• Reps. Ted Lieu and Obernolte introduced a separate AI bill to codify over 20 recommendations made by the House AI task force they previously chaired.

  • The bill would formally authorize CAISI, among other things.

• Sen. Jim Banks introduced the Senate version of the AI Overwatch export control bill.

  • Unlike Rep. Brian Mast’s House bill, the Senate version does not let Congress block chip exports — though it does still include a ban on Blackwell exports.

  • Rep. Mast pushed GOP leadership to hold floor votes on the Overwatch Act and other export control bills, to build momentum for inclusion in the must-pass NDAA.

• The House Homeland Security Committee and China Select Committee sent letters to Airbnb and Anysphere asking how they use Chinese AI models.

  • Sens. Chuck Grassley and Banks demanded answers from OpenAI, Anthropic, xAI, and other AI firms about employees with China ties accessing AI systems.

• Florida Speaker Daniel Perez blocked Gov. Ron DeSantis’ AI regulation bill, which advocates and the governor cast as him giving in to tech lobbying.

  • Perez was recently endorsed by the Leading the Future super PAC.

• Maine Gov. Janet Mills vetoed what would have been the nation’s first data center moratorium.

• EU countries failed to reach a deal on changes to the EU AI Act. Talks resume next month.

• China ordered Meta to unwind its $2b Manus acquisition.

INFLUENCE

• Sen. Bernie Sanders held an event to discuss the existential risks of AI and the possibility of US-China cooperation.

  • It received intense backlash from Republicans criticizing him for inviting two Chinese scientists to Zoom into the event.

• The WSJ covered the Trump administration’s efforts to kill AI legislation in Republican states, including Florida, Utah, Nebraska, Missouri, Tennessee and Louisiana.

• Tech billionaire and crypto exec Chris Larsen said he’ll spend $3.5m supporting Alex Bores in New York.

  • Larsen called Leading the Future’s attacks on Bores “really despicable,” saying “they are trying to destroy and intimidate and send a clear message that if you do come up with clear guardrails, we are going to crush you.”

• The Meta and Google-backed super PAC California Leads spent $2.4m on California state legislative races in just four days, per the Tech Oversight Project.

• A Midas Project report alleged that a news site was publishing AI-generated content pushing industry talking points, linking the site to the Leading the Future super PAC.

  • Leading the Future said it was “not aware of this platform” and that it “appears that a third-party vendor used it without our knowledge.”

• The AFL-CIO Tech Institute and 40+ organizations called on Congress to establish federal AI guardrails protecting workers.

• The Institute for AI Policy and Strategy released a guide for frontier AI companies to report risks from internal model use.

• Americans for Responsible Innovation released a white paper laying out a process for ensuring human control of AI weapons and verification of AI systems before Pentagon contracts are awarded, following the Google-Pentagon deal.

• AI is shaping how policymakers form opinions.

INDUSTRY

Elon Musk vs OpenAI

The biggest show in town this week was Elon Musk’s $134b lawsuit against OpenAI.

• Musk claims that Sam Altman and Greg Brockman went back on their promise to keep OpenAI a nonprofit.

  • Of Musk’s original 26 claims, only two — unjust enrichment and breach of charitable trust — reached the jury.

• On Monday, Musk boosted the New Yorker investigation into Altman on X. “Calling him ‘Scam’ Altman is accurate,” he tweeted.

  • Ronan Farrow, one of the article’s authors, responded by sharing his 2023 reporting on Musk’s role in the US government.

• The trial began in earnest on Tuesday.

  • Musk said his argument was simply that: “No one should be allowed to steal a charity.”

  • OpenAI’s lawyer William Savitt put it differently: “Mr. Musk comes to this court claiming that promises were made to him…and broken … [but] we’re here because Mr. Musk didn’t get his way at OpenAI.”

  • Judge Yvonne Gonzalez Rogers asked Musk, Altman, and Brockman to stop tweeting about the case. “All of you try to control your propensity to use social media,” she said. “Perhaps you’ve never done that before.”

• On Wednesday and Thursday Musk was cross-examined.

  • He painted himself as an AI safety advocate throughout.

  • …but when asked whether he knew what a safety card is, Musk reportedly smiled and said, “Safety card? Why would it be a card?” He didn’t know about other important safety practices in the AI industry, either.

  • Musk also seemingly admitted that xAI distills OpenAI models.

• Some recommendations, for those following along:

  • The Verge is rounding up all the evidence presented in court.

  • MIT Tech Review’s Michelle Kim, a lawyer herself, is live-tweeting the trial.

  • NYT’s Mike Isaac is too (ft. some extra color).

OpenAI

• Microsoft and OpenAI revised their partnership.

  • OpenAI can now sell its products across any cloud provider, including Amazon and Google.

  • Microsoft will keep its license (non-exclusively) to OpenAI IP through 2032, and receive revenue payments through 2030.

  • The amendment nixed the clause that could have cut off Microsoft’s share of OpenAI’s revenue and IP rights after it achieved AGI.

• Amazon announced that it will soon offer OpenAI models via AWS.

• Sam Altman outlined five principles for OpenAI: democratization, user empowerment, universal prosperity through widespread AI access, resilience through iterative deployment, and adaptability.

  • The document says OpenAI expects to pause development to collaborate with governments and others working on AGI “to ensure that we have sufficiently solved serious alignment, safety, or societal problems.”

• Altman announced that OpenAI is rolling out GPT-5.5-Cyber to “critical cyber defenders” this week, and will work with government and “the entire ecosystem” on access to the model and security.

  • The same day, OpenAI published an action plan for “democratizing AI-powered cyber defense.”

• The Wall Street Journal reported that OpenAI missed its internal targets for revenue and new users.

  • OpenAI countered that the report was “prime clickbait,” and said it’s “firing on all cylinders.”

  • CFO Sarah Friar told Bloomberg that the company sees a “vertical wall of demand” for its products and “we feel like we’re beating our plan at the highest level.”

• OpenAI signed contracts for 10 GW of compute, three years ahead of schedule.

  • It’s reportedly reworked its $500b Stargate plan, scrapping data center projects in the UK and Norway and shifting from joint ownership to leasing capacity.

  • The changes have reportedly unsettled partners but helped it secure 8GW of compute.

• Analyst Ming-Chi Kuo reported the company is also developing an AI agent smartphone, expected in 2028.

• Seven families sued OpenAI for allegedly failing to report a school shooter’s violent ChatGPT chats to authorities.

Google

• Google signed a deal letting the Pentagon use its AI models on classified work, for “any lawful governmental purpose” — the same terms OpenAI and xAI agreed to earlier this year.

  • That morning, Google suddenly dropped out of a Pentagon prize challenge to create autonomous drone swarms.

• It invested $10b in Anthropic at a $350b valuation, with potential for another $30b.

• Alphabet’s Q1 earnings report exceeded expectations as it upped its 2026 capex forecast to $190b.

  • Enterprise AI tools reportedly drove a 63% increase in Google Cloud revenue.

  • Google reportedly plans to deliver TPUs to customers’ data centers.

• DeepMind introduced its AI co-clinician research initiative, which aims to develop AI tools that “extend clinicians’ reach while ensuring they retain judgment and control.”

Anthropic

• Anthropic’s latest funding round would value it at over $900b, potentially surpassing OpenAI.

  • It reportedly asked investors to submit allocations for a $50b round that could close within a fortnight.

• Goldman Sachs reportedly blocked its Hong Kong bankers from accessing Claude.

• Anthropic released new Claude connectors for creative tools including Adobe, Ableton, and Blender.

Meta

• Meta raised its capex outlook to $145b, spooking investors who worry that the company’s AI infrastructure spending won’t pay off.

• It signed a deal with Overview Energy for up to 1 GW of solar energy beamed from space to power data centers.

• It is about to unwind its Manus acquisition, after China banned the deal.

Others

• SpaceX is reportedly telling investors that Elon Musk has sole power to fire himself from his roles as CEO and chairman.

• AWS revenue grew 28% in Q1, its fastest rate in almost four years.

• Microsoft said its AI revenue exceeded $9.25b in the most recent quarter, which it attributed to increased demand for AI software and servers.

• Mechanize, an AI startup that wants to fully automate work, raised $9.1m at a $500m valuation. OpenAI board member Adam D’Angelo participated in the deal.

• David Silver, the creator of AlphaGo, launched Ineffable Intelligence with a $1.1b seed round, aiming to build self-learning AI using reinforcement learning.

• AI tools that automate ad creation and targeting are doing wonders for Google and Meta’s advertising revenue growth, the New York Times reported.

• Chinese AI companies are reportedly driving demand for Huawei’s new AI chip, after the release of DeepSeek V4 last week — but Huawei is struggling to meet demand.

• The FIDO Alliance, Google, and Mastercard are teaming up to set industry standards for validating and protecting payments carried out by AI agents.

MOVES

• Sarah Dolan Schneider joined Anthropic’s federal policy comms team from public affairs consultancy S-3 Group.

• Brian McMillan was promoted to head of US policy at the Computer and Communications Industry Association.

• Chris Hayduk joined OpenAI as a forward-deployed engineer specializing in life sciences.

• Shimon Whiteson left Waymo to run a new multi-agent learning team at DeepMind.

RESEARCH

• OpenAI acknowledged the goblins:

  • ICYMI: Instructions for Codex include a line commanding the agent to “never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.”

  • OpenAI explained that, in training for the “Nerdy” personality customization feature between GPT-5.1 and GPT-5.4, they “unknowingly gave particularly high rewards for metaphors with creatures. From there, the goblins spread.”

    • Lesson learned: reward signals can shape model behavior in strange and mysterious ways.

• FAR.AI red-teamed DeepSeek’s V4-Pro and found three jailbreaks — including one that circulated on social media after DeepSeek’s previous model release — that got V4-Pro to comply with nearly all harmful requests.

• UK AISI published further details of its GPT-5.5 evaluations.

  • Notably, AISI’s James Aung said these tests were not on GPT-5.5-Cyber.

• Anthropic released BioMysteryBench, a benchmark including 99 questions written by bioinformatics experts.

  • While each question has ground-truth solutions, some of them are difficult or impossible for human experts to solve, which researchers argue makes it a powerful science benchmark.

  • Mythos Preview solved nearly 30% of problems that human experts could not, largely aided by its superhuman knowledge base.

• An amateur mathematician used GPT-5.4 Pro to solve a 60-year-old Erdős conjecture with a single prompt — discovering a novel method that experts believe may have broader applications.

• Researchers at the Center for Democracy & Technology and MIT found that fine-tuning models can cause “safety drift,” where small tweaks sometimes lead to large shifts in behavior (and vice versa).

• Biohub, backed by Mark Zuckerberg and Priscilla Chan, announced a $500m Virtual Biology Initiative to create open datasets for AI-powered predictive models of human cells.

• A team of researchers working with the Internet Archive found that a third of websites created since the launch of ChatGPT were clocked as AI-generated by Pangram, an AI detection tool.

• Rob Wiblin debunked the viral “95% of AI pilots fail” statistic.

  • He argued that, if you read the MIT-affiliated report closely, it shows that 25% of pilots succeeded within six months and over 90% of workers regularly used AI tools.

BEST OF THE REST

• The New York Times profiled Dwarkesh Patel, making sure to describe his “weightlifting-enhanced physique” and “dense beard that friends call ’majestic’” in the first 100 words.

• The Atlantic imagined what would happen if the US government nationalized AI companies.

• Jasmine Sun covered Silicon Valley’s fear of being banished to the “permanent underclass.”

• Stanford undergrad Theo Baker wrote about the Stanford freshmen being groomed for tech leadership by VCs offering pre-idea funding and mentorships.

• Tech executives are beefing up their personal security as anti-AI sentiment rises, The Information reported.

• Robotics startup Eka unveiled a remarkably nimble claw, which moved so fluidly that Wired’s Will Knight described it as “a ChatGPT moment for the physical world.”

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

On Monday, Google signed a deal letting the Pentagon use its AI models on classified work, for “any lawful governmental purpose” — the same terms OpenAI and xAI agreed to earlier this year. Anthropic refused in February, sparking a supply chain risk designation and months-long legal battle.

The announcement came as a complete surprise to Google employees, especially as, according to one DeepMind researcher, senior management had repeatedly insisted Google wouldn’t cave to the Pentagon’s demands, and urged employees to trust that leadership would make the right call.

Subscribe now

Many employees have spent months begging company leadership to follow Anthropic’s lead. Over 100 urged Jeff Dean, chief scientist at DeepMind and Google Research, to refuse any military deal that crosses basic red lines around domestic mass surveillance and autonomous weapons. Earlier this week, more than 600 Google employees signed a letter to CEO Sundar Pichai protesting the company’s ongoing negotiations with the Department of Defense.

But Google didn’t even alert its workers when the deal was signed — they had to find out by passing news stories around group chats. The closest thing to an announcement was an innocuous note from Kent Walker, Google’s president of global affairs, buried in the middle of a standard internal newsletter sent on Monday. It shared that the company “strongly support[s] the consensus that has emerged in the field regarding lawful use of AI,” but didn’t mention that a deal would be made.

After The Information broke the news, Pichai and Dean remained silent — though both tweeted to celebrate Google Translate’s 20th anniversary the same day.

Some AI researchers did publicly express their deep frustration with the contract. While it technically states that Google’s AI models are “not intended for, and should not be used for, domestic mass surveillance or autonomous weapons (including target selection) without appropriate human oversight and control,” it also explicitly “does not confer any right to control or veto lawful Government operational decision-making.”

The wording suggests that Google has little if any ability to stop the Pentagon from using its AI tools for the most controversial activities that lab employees, and many more outside AI, strongly object to. As Institute for Law and AI senior researcher Charlie Bullock put it on X: “’Should not be used for’ is not the same as ’shall not’ or ’will not’ be used for. ’Should not’ imposes no enforceable obligation on the Pentagon.”

He added: “While OpenAI claimed that they had technical safeguards that would prevent their models from being used to cross red lines, Google’s contract appears to obligate Google to remove any technical safeguards that are preventing DoW from accomplishing some lawful purpose. Domestic mass surveillance and autonomous targeting can both be lawful under some circumstances.”

Some DeepMind employees have spoken out publicly. “The contract includes some meaningless weasel words to allow for PR spin, but it seems so blatantly stupid, readers should feel insulted by it (I do),” tweeted Andreas Kirsch, a research scientist at DeepMind. In the words of another research scientist, Alex Turner: “If OpenAI offered a fig leaf, Google said ‘imagine we offered a fig leaf.’”

The lack of comment from Google management comes even though some have been publicly critical of exactly this kind of deal. At the end of February in the midst of Anthropic’s Pentagon drama, Dean tweeted: “Mass surveillance violates the Fourth Amendment and has a chilling effect on freedom of expression. Surveillance systems are prone to misuse for political or discriminatory purposes.”

Google has a mixed track record of responding to employee outrage at how its products are deployed. In 2018, workers successfully convinced the company to pull out of Project Maven, which uses AI to analyze and act on drone surveillance footage. This work was eventually picked up by Palantir and other contractors — including, until recently, Anthropic.

But in the years since, being “woke” fell out of vogue, and, like many large corporations, Google began cracking down on activism. When in 2024 employees protested against Project Nimbus, a $1.2b contract supporting Israeli surveillance and military operations in Gaza and the West Bank, over two dozen of them were unapologetically fired.

This latest deal with the Pentagon provides another test of how much employees at the company, and in particular those working at AI labs, can influence how their work is put to use.

“I feel sorry for our comms teams. There are no excuses to explain this away,” Kirsch tweeted. “I do not understand how this is ‘doing the right thing,’ and I think this violates ‘don’t be evil’ quite clearly on many levels. I personally feel incredibly ashamed right now to be Senior Research Scientist at Google DeepMind and I wonder how I’m supposed to do my work today.”

Share

If you look past the cryptic AI billboards lining Highway 101, San Francisco is still a city shaped by civil disobedience. For decades, young people, queer people, and weirdos of all stripes flocked west and settled here, building co-living spaces and resisting the powers that be. There’s plenty of anti-establishment angst to go around.

On a partly sunny Saturday this March, protestors gathered outside Anthropic’s office to rally against the AI race, a few blocks southwest of Market Street, the city’s historic artery of dissent. AI slop coverage of the protest illustrated a dense crowd framing a pink-haired woman in front of a “SAVE OUR JOBS” banner, screaming into a megaphone. San Francisco stuff.

In reality, Stop the AI Race pulled between a few dozen and a couple hundred people — mostly men, very earnest, and nearly all white — carrying more esoteric signs: “IT’S SMART ENOUGH,” one said. “MAY YOUR GPUs CHIP AND SHATTER,” said another. “PAUSE IS DEMANDED if you aren’t CONSISTENTLY CANDID.” Despite the city’s appetite for nonviolent protests and growing antagonism toward AI companies, the entire crowd could have comfortably fit in a couple of BART cars.

Subscribe now

Then on April 10 a Molotov cocktail was thrown at Sam Altman’s San Francisco mansion sometime between 3am and 4am. Twenty-year old Daniel Alejandro Moreno-Gama was arrested for the attack outside OpenAI’s headquarters, where he was allegedly trying to break in. In his backpack, officers reportedly found a manifesto listing the names and home addresses of other AI executives. Earlier this year, he wrote Substack posts about death, destiny, and existential risks, or “x-risk,” posed by artificial intelligence.

(Within 48 hours, two others were arrested for shooting at Altman’s house before being released pending investigation — they reportedly had no connection to Moreno-Gama.)

While AI-informed types were quick to condemn the violence, many outside the Silicon Valley bubble seemed thrilled. “One does have to admire the skills of someone who can pour a good cocktail in this weather,” someone posted to Reddit. Instagram users — many with full legal names publicly displayed in their profiles — reacted similarly. “Where can we support their bail fund? ✨” one said. “New love language just dropped 😍,” replied another.

The vibe mismatch between the AI crowd and outsiders was unsettling, but similar splits over violence against corporate targets have happened before. In December 2024, #FreeLuigi went viral as users — mostly young people — painted Luigi Mangione as a folk hero after he killed UnitedHealthcare CEO Brian Thompson. The handsome suspect became the protagonist of a buzzy musical and dozens of steamy fanfics. Fans even bought merch.

Silicon Valley loves to say we can “just do things,” but when it comes to meaningfully changing the arc of AI development, most of us can’t do anything at all. Committing violence against tech CEOs and their families is not forgivable, but it’s certainly agentic. To a radicalized young man who believes that if anyone builds superintelligent AI, everyone dies, burning down a CEO’s house and company headquarters to save the human race might seem like the lesser of two evils. After all, if a trolley is approaching a fork in the tracks, utilitarian logic says you should pull a lever to send it running over a single victim, if it would save many others who’d otherwise die as a consequence of your inaction.

Over the last couple years, Moreno-Gama reportedly posted 34 messages to anti-AI activist group PauseAI’s public Discord under the Dune-inspired handle “Butlerian Jihadist,” referencing the book’s fictional crusade against thinking machines. PauseAI, whose US branch founder Holly Elmore gave a speech at last month’s Stop the AI Race protest, stated that it “unequivocally condemns this attack and all forms of violence, intimidation and harassment,” and that “violence against anyone is antithetical to everything we stand for.”

This attack and its aftermath — celebrated by anti-AI normies, rebuked by AI safety insiders, motivated by the same “doomer” canon that inspired Altman to found OpenAI in the first place — may have been a foreseeable consequence of a technical, existential-risk-focused community sounding the alarm before building a broadly-appealing coalition to channel people’s anxiety and anger into political action.

At least for now, protests against future superintelligence, focused on hedged demands such as “CEOs should commit to pause AI development if everyone else does too,” can only rally so many people. But even those who don’t spend time on LessWrong love Gen Z dudes who attack CEOs. This narrative transcends AI safety, argued tech journalist Brian Merchant: it “identif[ies] billionaire AI executives as uniquely powerful actors, who are all but unaccountable to democratic constraints and society’s best interests.”

While it’s easy to dismiss memes and Instagram comments as nihilistic noise, the legitimate frustration behind them is more widespread than the AI safety community seems to realize. A recent NBC News poll found that, netting positive views against negative ones, more voters feel worse about AI than about ICE (i.e., really bad). Mainstream anxieties about job loss, cyberattacks, and mass surveillance — which all rank relatively high on the public’s list of concerns about what AI might do — tie into x-risk-pilled concerns such as gradual disempowerment and loss of control.

The AI safety community has historically worried that addressing normie concerns would come at the expense of x-risk, and possibly knock it off potential legislation altogether. But these pressing, present socioeconomic issues may be the gateway that gets x-risk on the table. “These are the things that people are feeling right now,” said Alex McCoy, Head of Left Coalition at political advocacy group Humans First. “It doesn’t mean that they don’t believe in Skynet.”

Politicians respond to what they think their constituents want, and the vast majority of Americans do not want AI to continue along its current trajectory. The momentum is there — people are beginning to take action, however imprecisely, driven by deeply-rooted feelings of unfairness and demoralization. Traditional AI safety advocates may just need to cede enough control of their narrative to harness it.

It all comes down to insularity

Traditionally, the work of AI safety has excluded the public by design. For years, AI safety discourse has largely unfolded behind closed doors, between researchers, executives, and the policymakers they have direct access to. This has often been prioritized over broader public engagement, for what seemed like good strategic reasons. Long, information-dense blog posts, closed-door meetings and money carry more weight in this world than mainstream media, anyway. Why waste time explaining “AGI” to normies, when they’re not drafting policy proposals?

The AI safety field was built on the idea that, with enough compute, money, and brainpower, a small group of very smart people can save the world — no help, input, or permission required. They reasoned that preventing tech companies from building a deadly machine god should be done as quietly as possible, without attracting the kind of public or political attention that might inadvertently spark an ill-fated race towards superintelligence. In hindsight, this wasn’t paranoid: philosopher Nick Bostrom’s 2014 bestseller Superintelligence: Paths, Dangers, Strategies surfaced concerns about x-risk from the depths of the rationalist blogosphere to the New York Times best seller list — and partially motivated OpenAI’s founding.

But in choosing to operate largely behind the scenes, the AI safety community created a vacuum that’s now being filled by industry lobbyists, populist politicians, and radicalized individuals. Leading the Future, a pro-AI super PAC network backed by venture capitalists and AI executives, has reported raising more than $75m, and claims to have raised $140m. Meanwhile, Bernie Sanders, Gen Z influencers, and data center NIMBYs are leading the populist backlash against the industry. Existential risk has only very recently entered the conversation, and it comes with baggage.

While AI safety is not the same thing as effective altruism, the two are deeply entangled. EA, a movement that attempts to maximize human flourishing through quantitative reasoning, funneled a lot of talent and money toward early AI safety research. Today, many of the field’s biggest names speak at EA conferences, share donors and shape AI policy decisions. At this point, it’s very hard to disentangle the public’s perception of “AI safety” from EA itself.

The community’s manufactured insularity and inclination for “Secret Congress”-esque policymaking stoked an aura of opacity, leading to public distrust from all sides. “In Silicon Valley, the EAs are viewed as one step to the right of Elizabeth Warren,” a former Biden official told Politico. “Conversely, in DC, on the left they think EAs are the devil.” Everywhere else, the public’s response is mostly “Who are you guys?,” said Akshyae Singh, co-founder of The Frame, an accelerator for AI safety content creators.

This problem isn’t completely lost on insiders. Two years ago, a survey asked 17 prominent AI safety experts what big mistakes the AI safety community was making. Their biggest gripes: “overly theoretical argumentation” and “being too insular.” A couple of respondents explicitly called out the EA urge to dismiss public outreach in favor of technical problem-solving as the root of the problem. Richard Ngo, an independent researcher who previously worked at DeepMind and OpenAI, argued that, at least at the time, AI safety overemphasized fundraising and back-room deals at the expense of the field’s public image. “From the perspective of an external observer,” he responded, “it’s difficult to know how much to trust stated motivations, especially when they tend to lead to the same outcomes as deliberate power-seeking.” Another respondent, METR researcher Daniel Filan, said, “If the three biggest oil companies were all founded by people super concerned about climate change, you might think that something was going wrong.”

Others, including prominent commentator Anton Leicht, reject the idea of building a popular movement altogether. One of Leicht’s primary concerns, echoed across the upper echelons of Silicon Valley, is that in the process of trying to address everyone’s concerns, a populist AI safety movement will wind up dropping the one thing that ought to be its central concern: existential risks. The line items that rally a crowd — data center moratoriums, for instance, or child safety legislation — don’t necessarily make for great policy. Leicht believes that the AI safety community’s strongest assets are high expert credibility, and the fact that most people seem to vaguely support AI regulation. He worries that a poorly-executed popular movement could ruin both.

Even so, Leicht acknowledges that there are tradeoffs to playing inside baseball. “There is a huge epistemic gap between the small community of people who think they’re on the inside, and the rest of the world,” he told me. Like many others in the field, he’s wary of communicating to the “lowest common denominator” about existential risk without feeling confident that non-experts have enough background knowledge to contribute productively. “I don’t have a solution to it,” he said. “I think people are just not that good at it.”

McCoy described this perspective as “emblematic of the sort of anti-politics” of the AI safety community. Congress will address AI one way or another, he said, and whatever organized constituencies show up will shape what that legislation will ultimately look like.

“If the AI safety community does not take seriously the necessity to engage in capital-P politics,” he cautioned, “its concerns will be left out.”

Who’s missing?

One big problem, Singh argues, is that “people in this field don’t tend to do most things unless there’s absolute concrete proof” that it will be effective. In the absence of a previously-successful effort to increase demographic diversity within the AI safety community, there are no hard numbers saying that decentering the concerns of a tiny, homogenous group of people will make passing AI regulation easier. But a mass movement with the power to pressure governments and AI companies, by definition, has to include a lot of people — including those who don’t currently fit inside the AI safety bubble.

In the world of left-wing community organizing, there’s a common refrain: center the most impacted. “That’s not out of some kind of virtue signaling,” McCoy said. “It’s because those are the people that are going to fight the hardest.” But those most directly feeling the real-world impacts of AI today — first-generation college graduates struggling in an increasingly-nonsensical job market; women humiliated by nonconsensual deepfakes; content moderators in Nairobi exposed to traumatic content for a couple dollars an hour — are those least represented within the AI safety community.

And there is data to back this up. A 2025 Seismic Report, for instance, found that women are over twice as concerned about AI than men. On a global level, a UN report found that female-dominated occupations are almost twice as likely to have high automation potential than male-dominated ones, And, relative to men, they’re over three times more likely to have their jobs disrupted by AI. Yet, women are few and far between in the AI safety field, particularly in technical roles — perhaps in part because near-term socioeconomic concerns often wind up lower on the priority list than more abstract concerns about alignment, interpretability, and the governance of AI systems that don’t exist yet.

The dominant AI safety discourse “primarily serves the interests of technological institutions and stakeholders in high-income nations, often privileging abstract future scenarios over pressing sociotechnical harms that disproportionately affect marginalized communities,” the Brookings Institution wrote last year. Growing up in India, this deeply frustrated Singh. “If brown people don’t have a voice, there is no way in hell that you’re making a solution that benefits me,” Singh said. “Like, how do you know how I feel?”

Getting abstract future scenarios onto the policy agenda may, perhaps counterintuitively, depend on addressing the concrete harms already shaping people’s lives — and, by extension, informing who they vote for and which AI products they use and pay for. The AI safety community already tried relying on a handful of well-connected experts to regulate the AI industry behind closed doors, and it’s not working.

The AI safety field is mostly talking to itself, and it’s created an information void that’s being filled by populist anger. Outside Silicon Valley, most people don’t experience AI as a powerful coding tool or existential threat. Rather, it’s a symbol of the machine we’re meant to be raging against — not an extinction risk, per se, but something billionaires are using to forcibly strip humans of their humanity.

“I don’t think it’s about persuading people that superintelligence is bad,” said John Sherman, president of the AI Risk Network. “It’s about persuading people that they can make a difference.”

How to (hopefully) not screw up the AI safety movement

Sherman proudly introduced himself to me as a Baltimore resident who, until a couple years ago, “didn’t know anything about AI.” After decades of working in TV and video production, “I can edit in Adobe Premiere,” he joked. “That’s about as technical as I get.”

Then he stumbled across Eliezer Yudkowsky’s 2023 article in TIME Magazine, in which he wrote: “If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.”

Today, Sherman also runs a nonprofit called GuardRail Now, focused on communicating AI x-risk to normies. “My primary concern is x-risk,” he said, but “I think we have to take side roads to get to the destination. And a lot of people in AI safety are unwilling to consider that — but they’re not getting anywhere. So like, how’s it going? You’re stuck.”

Sherman’s theory of change is simple: people are feeling a visceral sense of unfairness. AI companies are reshaping society without anyone’s consent, so “we need to make unsafe AI bad for business.” This can be framed in terms of existential risk, he said: “We’re building systems that we don’t know how to control, that we don’t understand how they work, that the experts say can kill everybody. Why would we do that?”

Imagine a family in Ohio, where everyone is experiencing AI differently. Dad is uneasy about new AI policies at his white collar job, and a data center is being built behind the neighborhood school. His teenager is doing god-knows-what on ChatGPT, and his college kid is talking about how she wants to drop out and work in construction. “To build a real movement,” Sherman said, AI safety advocates “need to run full speed ahead towards people who are concerned about their kids, towards people who are concerned about data centers … the whole thing, all of it.”

This approach makes many in the AI safety community uneasy, reasonably so. A little over two weeks after hiring Sherman as its Director of Public Engagement, the Center for AI Safety parted ways with him after clips surfaced of Sherman telling podcast listeners that the “proper reaction” to the AI race was to “walk to the labs across the country and burn them down.” CAIS announced that the “connotation of statements like this do not reflect CAIS’s values,” to distance itself from this kind of fiery rhetoric — which, while in this case hypothetical, could radicalize people who may already be angry enough to act. (Sherman said he regretted using the language, clarifying that he meant “when the general public finds out their lives are being risked for AI, the reasonable reaction is to shut it down.” However, he has continued to describe AI in rather intense, hyperbolic terms.)

But building a movement doesn’t mean encouraging violence. The climate movement, for example, managed to grow from scientists expressing concern amongst each other to a global issue, without linearly increasing the risk oil executives faced from potential attackers. Decades of climate activism, including conveying the existential stakes of climate change to the public, hasn’t led to the kind of violence one might expect if x-risk messaging were a reliable radicalizer. Arguably, political organizing gives individuals a more structured outlet for their righteous frustration than attempted murder. The alternative — an AI safety community that stays silent while populists take up space around them —opens the door to unstructured acts of radical violence and worse policy.

Even Yudkowsky, Machine Intelligence Research Institute (MIRI) president Nate Soares, and the broader Berkeley rationalist scene they helped build — who have arguably shaped the conversation around existential risk more than anyone — recently pivoted hard toward public outreach. Soares, who co-authored If Anyone Builds It, Everyone Dies, has been on tour for the book. Last month, he spoke at the Stop the AI Race protest in San Francisco.

In February, Soares and Yudkowsky sat down with Bernie Sanders at MIRI’s Berkeley offices. After decades of treating persuading the public at large as a distraction, some of the biggest players of the inside game have, however begrudgingly, realized they can’t do this alone.

It’s not an accident that Sanders was the first to get there. Perhaps more than anyone else in DC, his politics is centered around pushing back against unaccountable billionaires and their friends in government — which, stripped of jargon, is what preventing superintelligence ultimately requires. Given the Trump administration’s damage to American soft power abroad and the unimaginable amount of money AI companies have to throw around, an international treaty to pause the race looks relatively unattainable.

But AI companies are still businesses. And the people with the most leverage over large corporations are customers, workers, investors and voters — not researchers writing alignment papers.

“AI safety as a persuasive cause will never have more power than the industry’s hard power in dollars and political influence,” McCoy said, “unless it is allied to constituencies who can lend their power together.” Conversations about extinction risk among rich tech guys in San Francisco, he added, are “not the message that is going to get hundreds of people to show up to a protest.” The protests that matter will be about jobs, surveillance, kids, data centers — what McCoy calls the “symptoms.” But the disease, in his framing, is exactly what x-risk advocates have been trying to address, seen from another angle: “an unaccountable set of billionaire investors and executives who have no guardrails and are seeking to concentrate an incredible amount of power in their companies.”

One could argue (and Leicht does) that by making AI safety an “omnicause” addressing everyone’s prosaic concerns, it will get elbowed out of whatever legislation ends up passing. But existential risk is already last on the public’s list of AI concerns across every demographic, according to last year’s Seismic poll. It might need to join a coalition of other causes to get on the table at all.

Subscribe now

“AI safety right now is islands,” Sherman told me. “We need an ocean to connect the islands.”

Silicon Valley speaks its own language, and most people outside the bubble don’t understand it. “The best thing that the AI safety movement could do would be to build an army of surrogates who are regular people, going into their own communities and talking about this stuff,” Sherman said — “not strangers from a foreign land speaking a different language.”

Singh agrees. The catastrophic threats posed by AI aren’t hard to grasp. “Like, my dad, my mom — I can explain it to a lot of people, and they’ll get what’s going on,” Singh said. But they’re often made unintelligible by people who treat communicating to people without technical backgrounds as an inconvenience rather than a necessity. So Singh launched the Frame Fellowship, an eight-week incubator for content creators to bring AI safety discourse to the masses via YouTube, TikTok, and Instagram (Michaël Trazzi, who organized the Stop the AI Race protest, was also a fellow).

Long-term existential concerns about AI aren’t separate from near-term populist anxieties. Loss of control and gradual disempowerment are natural extensions of power concentration and job displacement. When microinfluencers talk about AI in “Get Ready With Me” TikToks, or neighborhoods band together at town hall meetings to voice their concerns, the case for existential risk becomes the endpoint of what people are already afraid of.

The AI safety community has a window of opportunity to make its case to the broader public, and some already are. While accelerationists such as Marc Andreessen have dismissed efforts to communicate about AI risks as “propaganda” fueled by “unaccountable dark money,” pro-industry leaders are also investing in communicating beyond the Bay Area bubble. The majority of people in the US feel uncomfortable about the current trajectory of AI, and this discomfort will likely turn into action. Whether that manifests as voting power or bottles of gasoline flying over San Francisco depends on building a movement.

Share

And a reminder: applications close this Sunday for our Head of Audience role. If you’d like to own Transformer’s growth strategy, make sure to apply.

NEED TO KNOW

• DeepSeek released its V4 model, which it says is three to six months behind the performance of the leading frontier models.

• Chris Fall will reportedly be the new director of the Center for AI Standards and Innovation.

• Rep. Sam Liccardo said he won’t co-sponsor Rep. Jay Obernolte’s forthcoming AI bill.

But first…

THE BIG STORY

OpenAI’s newly released GPT-5.5 is, according to the UK’s AI Security Institute (AISI), the world’s most capable model on individual cyber tasks, and can complete a “32-step corporate-network attack simulation estimated to take an expert 20 hours.” It appears to be similarly capable (if slightly worse) at carrying out a cyberattack as Anthropic’s unreleased Mythos.

But unlike Anthropic, OpenAI is making a version of GPT-5.5 available to the general public. Rather than restricting access to the model altogether, OpenAI hopes to restrict the use of particularly dangerous capabilities through its safety stack — making the model refuse concerning cyber requests from normal users, and only allowing such requests from those vetted under its “Trusted Access” program.

Yet we have no idea if that safety stack is good enough. And we have reason to believe that it might not be. Alongside its cyber testing, AISI also tested OpenAI’s safeguards, and “found a universal jailbreak with six hours of expert red teaming.” Such a jailbreak would let users circumvent OpenAI’s safeguards, giving them access to the powerful — and, in the wrong hands, dangerous — cyber capabilities. OpenAI claims to have addressed the issue, and says its own external red-teaming campaigns confirmed that the final launch configuration blocked all verified high-severity cyber jailbreaks. But, crucially, AISI — a trusted third-party evaluator with immense technical expertise — was not able to properly run tests “to verify the effectiveness of the final configuration.”

In other words: we do not know if GPT-5.5 is actually safe to release. All we have to rely on is OpenAI’s word.

Such a situation may have been acceptable in 2023. In 2026, with models posing genuine risks to national security and plenty of other vital systems, it no longer is. It is laudable that OpenAI and other companies allow AISI, the US’s CAISI, and third-party evaluators to perform pre-deployment evaluations. But if those organizations are unable to actually verify if a model is safe to release — and if a company has no obligation to listen to them — the exercise is limited.

This is not just an OpenAI problem. If Anthropic wanted to go down the same route, nothing would stop it. And as this week’s Mythos leak showed, its own security practices do not appear up to the task of safeguarding dangerous capabilities.

None of this is to say that GPT-5.5 is dangerous. OpenAI’s updated safeguards may in fact be robust to jailbreaks, and other aspects of its safety stack (such as monitoring users’ requests and banning accounts that raise too many red flags) provide an extra level of security. The point is that we are currently at the mercy of a private company grading its own homework, and all the frontier labs making the final call on what and when to release. Given the potential consequences of unsafe releases, that is no longer acceptable.

GPT-5.5 might be totally safe to release. It also might not be. Neither OpenAI, nor Anthropic or any other frontier developer, should be the one who gets to decide.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• AI safety PACs should be more transparent about who’s funding them — Veronica Irwin asks why Public First Action isn’t disclosing all its donors.

Subscribe now

THE DISCOURSE

Sam Altman shared Mythos takes on Core Memory:

• “If what you want is, ‘we need control of AI, just us, because we’re the trustworthy people,’ I think fear-based marketing is probably the most effective way to justify that.”

• “It is clearly incredible marketing to say, ‘We have built a bomb. We are about to drop it on your head. We will sell you a bomb shelter for $100m.’”

Trump said Anthropic is led by “high IQ people” on CNBC:

• “We’ll get along with [Anthropic] just fine … I think they can be of great use.”

roon (sort of) praised Anthropic, too:

• “Claude is an excellent product and it bodes well for [Anthropic] that their main problem is everyone really wants it and so they have to do odd shit to shake off demand.”

E/acc anon bayes thinks the AI industry has failed to articulate positive futures:

• “If the singularity is making it hard to see the future, people’s instinctive reactions will take over. For most people that default is fear.”

• “Many people in tech are in a shameless state of defection. Trampling each other on the way to the lifeboats is not a belief system. If we want a future of human flourishing for us and our descendants, we will have to make it so by fighting against the many powerful forces at odds with this goal.”

Bill Maher dedicated a segment of his show to P(doom):

• “I get it. [AI] can do some shit. Still, at the end of the day, you’re selling your humanity for bar tricks. I mean, what was the plan? Just create an all-powerful, self-sustaining super intelligence that can out-think us and then see what happens?”

• “We’re letting a handful of hoodie-wearing, on-the-spectrum sociopaths, practically robots themselves, roll the dice on species extinction … even these guys are afraid of what they’ve created.”

Helen Toner testified at a Senate hearing that “beat China!” isn’t a great plan:

• “The winner of any AI race between the US and China is the AI.”

• “...it is very important that the US AI sector remains ahead of the Chinese AI sector, but if that’s at the expense of AI overrunning the entire planet, then that is, you know, that hasn’t benefited us.”

Palantir posted a manifesto of sorts:

• “The engineering elite of Silicon Valley has an affirmative obligation to participate in the defense of the nation.”

• “If a US Marine asks for a better rifle, we should build it; and the same goes for software.”

Zachary Jones criticized the Public First super PACs’ strategy:

• “Public First has repeatedly intervened in favor of moderate Democrats against progressive opponents who also back comprehensive AI regulation. This has had the effect of polarizing the left against the AI safety community, limiting the capacity of experts to influence outcomes of the emergent wave of anti-AI populism.”

POLICY

• Trump said a deal with Anthropic for Department of Defense use is “possible” despite the Pentagon labeling the company a supply chain risk.

  • The comments came after Dario Amodei met with White House Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent.

  • In the meantime, the NSA was revealed as yet another agency reportedly using Anthropic’s Mythos Preview model despite the supply chain risk designation.

  • CISA, however, reportedly lacks access to Mythos.

  • The DC Circuit judges who denied Anthropic’s request to temporarily limit the supply chain risk designation kept the case to decide it on the merits.

• Chris Fall will reportedly be the new director of the Center for AI Standards and Innovation. He was previously director of the Department of Energy’s Office of Science.

  • Former OpenAI and Anthropic researcher Collin Burns was reportedly lined up for the role, but the Commerce Department changed its mind “while Burns was in the onboarding process,” according to the Daily Signal’s Elizabeth Mitchell.

• It was a busy week for export controls, with a slew of bills advancing out of the House Foreign Affairs Committee.

  • That included the MATCH Act, which would pressure allies to stop selling semiconductor manufacturing equipment to China, and the AI Overwatch Act, which would block Nvidia Blackwell chip sales and give Congress veto power over H200 licenses.

    • Micron is reportedly lobbying Congress to pass the MATCH Act — but the bill is reportedly creating tensions between the US and allies such as the Netherlands, where ASML opposes the bill.

    • Rep. John Moolenaar, who chairs the House China Committee, called for US-Dutch coordination.

    • Meanwhile, the House Foreign Affairs Committee’s top Democrat, Rep. Gregory Meeks, warned that the MATCH Act could damage US-allied relations and trigger Chinese retaliation, despite his support for the bill.

  • Rep. Moolenaar also introduced the SCALE Act, which would establish export controls on advanced semiconductors to China based on a rolling technical threshold tied to adversaries’ chip production capabilities.

  • Commerce Secretary Lutnick said Nvidia has not yet sold its H200 AI chips to China anyways, citing a lack of permission from the Chinese government.

• OSTP director Michael Kratsios said the US has evidence that China is running “industrial-scale distillation campaigns” on US models.

  • He said the government will work with companies to prevent this, and “explore a range of measures to hold foreign actors accountable for industrial-scale distillation campaigns.”

  • (One of the bills advanced by HFAC this week was the “Deterring American AI Model Theft Act”.)

• Rep. Jay Obernolte said he is “close” to releasing a comprehensive federal AI regulation proposal that would preempt state laws and regulate AI use in specific sectors like healthcare. He also said it would be “hundreds” of pages long.

  • Rep. Sam Liccardo said he won’t co-sponsor the bill because it doesn’t have “critical requirements” to “ensure that there is a race to the top, to safety.”

• Sen. Marsha Blackburn said she’ll be “pushing forward” with her AI bill this fall.

  • The bill received a range of new endorsements this week.

• Rep. Blake Moore introduced a bill to ban AI chatbots in children’s toys, prompting criticism that it would cut off educational AI tools.

• Florida sent criminal subpoenas to OpenAI after a shooter used ChatGPT to plan a mass shooting at Florida State University.

• Maine passed a bill banning large-scale data center development — forcing Gov. Janet Mills, running in a tough Senate primary race, to decide whether to sign it.

• China is reportedly restricting AI firms from accepting US investment without government approval, in response to Meta’s acquisition of Manus.

INFLUENCE

• Tech giants, including AI companies, reportedly spent a combined $20m on lobbying in Q1 2026.

  • Anthropic spent $1.56m, a 333% increase from Q1 2025.

  • OpenAI spent $1m, an 82% increase from Q1 2025.

• Pro-AI safety policy group Public First Action is reportedly endorsing six House Democrats for the midterms, including Reps. Don Beyer and Brad Sherman.

• NY-12 candidate Alex Bores proposed a range of policies to tackle AI-driven unemployment, including an “AI dividend” funded by a token tax and equity stakes in frontier AI firms.

  • NY Mag interviewed politicians about their thoughts on AI job displacement, including Sen. Elizabeth Warren and Sen. Josh Hawley.

• Hawley urged Republicans to refuse money from pro-AI super PACs, saying there’ll be a “political cost” for failing to regulate AI.

• The Rockefeller Foundation announced a $100m initiative to help US workers adapt to AI-driven job displacement in 250 communities.

• AVERI published an analysis of audit-related AI legislation in the US and endorsed Illinois bill HB 4705/SB 3261, which builds on SB 53 and the RAISE Act by verifying compliance with companies’ safety policies.

• A cross-faith coalition urged Congress to pass safeguards on AI-enabled weapons.

• Dario Amodei is co-hosting a party at Cannes Film Festival with ex-Vanity Fair editor Graydon Carter and CAA’s Bryan Lourd.

• a16z announced an investment in MTS, a new media company seemingly aiming to take TBPN’s crown.

• AI safety advocates are partnering with social media influencers to warn about AI extinction risks. ControlAI alone reportedly spent $100,000 monthly on content creation.

  • Online personality and sex researcher Aella is running “PLZDONTKILLUS,” a residency for video creators. Applicants are asked “If you had to have sex with a cow would you rather it be dead or alive?”

INDUSTRY

DeepSeek

• DeepSeek released V4, which it claims is the world’s most powerful open-source model — offering competing performance with US closed-weight models while being more efficient.

  • The model was released in a V4 Pro version with 1.6T parameters and V4 Flash, with 284b. Both come with a 1m token context window.

  • It reportedly runs at lower cost than leading US closed-weight competitors, but the company conceded that it was three to six months behind the performance of the leading frontier models.

  • But the efficiency only goes so far: DeepSeek said that “due to constraints in high-end compute capacity, current service capacity for Pro is very limited.”

  • Council on Foreign Relations’ Chris McGuire suspects that the lack of detail on how the model was trained suggests it was trained on banned Nvidia Blackwell chips.

• DeepSeek is fundraising for the first time to try to prevent researchers from defecting to rivals such as ByteDance and Tencent.

OpenAI

• GPT-5.5 appears to be the best publicly-available model to date.

  • It seems to be particularly good at helping researchers with AI R&D, though chief research officer Mark Chen said “full end-to-end research” capabilities were “a couple of years down the line.”

  • It’s also very good at virology: SecureBio said the model “can provide wet-lab virology troubleshooting assistance above expert level.”

  • OpenAI launched a Bio Bug Bounty for universal jailbreaks that defeat its biology safeguards.

• OpenAI launched ChatGPT Images 2.0 — it’s impressive.

  • The updated model can create multiple consistent images with a single prompt, search the web, and (mostly) get text right.

  • Enjoy these Anthropic and OpenAI-themed ‘Where’s Waldo’ images, courtesy of Jeffrey Ladish.

• It also launched ChatGPT for Clinicians, a free version of ChatGPT for verified US medical workers, and HealthBench Professional, which evaluates clinical tasks.

• It announced an intensified cybersecurity collaboration with Microsoft, where OpenAI will give Microsoft access to its most capable models through Trusted Access for Cyber.

• It reportedly briefed government agencies and Five Eyes allies about GPT-5.4-Cyber.

• It introduced Codex-powered workspace agents in ChatGPT, designed to run team workflows, and Chronicle for Codex, which uses screen captures to build contextual memories.

• It also introduced Privacy Filter, an open-weight model that runs locally to mask personally identifiable information in text.

• It has reportedly pledged up to $1.5b to a $10b joint venture with private equity firms to deploy AI tools in their portfolio companies.

• SoftBank is seeking a $10b loan secured on its OpenAI shares, adding to its mounting debt.

Anthropic

• Anthropic expanded its Amazon partnership to get up to 5 GW of compute for Claude and a $5b investment, with up to another $20b planned for the future.

• Central banks and intelligence agencies outside the US are worried that, by limiting Mythos access to US organizations (and the UK’s AISI), they’ve been placed at a geopolitical disadvantage.

  • Anthropic is reportedly planning to grant Mythos access to European and UK banks soon.

  • Meanwhile, unauthorized users reportedly accessed Mythos via a third-party Anthropic contractor on a private Discord channel.

• Anthropic’s valuation rose to as much as $1t on some secondary markets such as Forge Global.

• It admitted that Claude Code has been worse recently, blaming bugs that have now been fixed.

• It partnered with Freshfields to build specialized AI tools to help attorneys with their legal work.

• It started requiring ID verification for some users, in an effort to crack down on unwanted usage in countries such as China, Russia, and North Korea.

Google

• Google DeepMind launched Deep Research and Deep Research Max, agents that create fully-cited reports from both web search and custom data, including internal docs.

• Google assembled a “strike team” to make its AI coding models more competitive with Claude Code and Codex.

  • It said that 75% of its new code is AI-generated before review by engineers, up 25% from a year and half ago.

• Google Cloud announced its next-gen TPUs, tailored for creating AI software and inference.

• It’s in talks with Marvell to make new AI inference chips — a memory processing unit and a TPU.

• It released Decoupled DiLoCo, which lets distributed “islands of compute” run asynchronously to prevent hardware failures from stalling training runs across multiple data centers.

• It signed a multibillion-dollar cloud deal with Thinking Machines Lab.

• YouTube opened its AI deepfake detection tool to celebrities, athletes and public figures to flag and request removal of unauthorized uses of their likeness.

SpaceX

• SpaceX partnered with Cursor to build AI tools for coding and knowledge work.

  • The deal reportedly allows SpaceX to either pay $10b to Cursor, or eventually acquire the company for $60b, depending on how well the arrangement goes.

  • Microsoft reportedly considered buying Cursor, but didn’t make an offer.

• SpaceX’s debt rose to $23b last year, largely due to an AI infrastructure lease for xAI.

• Its focus has notably shifted from colonizing Mars to building AI data centers in space in the lead-up to its IPO, the New York Times reported.

  • But its pre-IPO filing warns that space-based data centers rely on “unproven technologies, and may not achieve commercial viability.”

• Its S-1 filing reportedly estimates that SpaceX’s total addressable market could be up to $28.5t.

  • But it reportedly warns that investigations into Grok’s generation of nonconsensual explicit imagery could lead to loss of market access.

Meta

• Meta reportedly started capturing employee mouse movements and keystrokes to train AI agents on work tasks. Employees predictably hate this.

• It will reportedly lay off about 10% of its staff next month — with more cuts planned for later this year — as part of a push towards a more AI-driven workforce.

• It partnered with AWS to deploy “tens of millions” of Graviton CPU cores for agentic AI workloads.

• It announced a free month-long program to train people to be fiber technicians for data center construction sites.

Microsoft

• GitHub is raising Copilot prices, restricting Claude usage to its most expensive subscription tier.

• Microsoft will invest $17.9b in Azure AI infrastructure in Australia by the end of 2029.

• It partnered with North America’s Building Trades Union to offer free AI literacy courses and industry credentials to upskill construction workers.

Others

• TSMC said it plans to open an advanced chip packaging plant in Arizona by 2029 to address AI chip supply bottlenecks for Nvidia and others.

  • The chip maker said it was delaying buying ASML‘s high-NA EUV lithography machines until at least the end of 2029, citing their €350m ($410m) cost as “very, very expensive.”

• Cohere agreed to acquire Germany’s Aleph Alpha in a deal valuing the combined group at about $20b, creating a transatlantic company focused on ‘sovereign’ AI systems.

• Moonshot AI released Kimi K2.6, an open-source model with powerful coding capabilities that can coordinate up to 300 sub-agents across 4,000 steps.

  • It’s notably expensive at $0.95/$4.00 per 1m input/output tokens.

• Core Automation, founded by ex-OpenAI VP Jerry Tworek, announced its launch.

  • Its objective: build “the world’s most automated AI lab.”

• Sooth Labs, founded by ex-Meta employees and backed by Yann LeCun and Jeff Dean, is raising about $50m to build forecasting models.

• Recursive Superintelligence raised $500m at a $4b valuation to build self-improving AI. (The concept is still reportedly at the research stage.)

• Jeff Bezos’s Project Prometheus is close to raising $10b at a $38b valuation to build AI that understands the physical world.

• Cognition is reportedly in talks to raise funding at a $25b valuation, more than doubling from $10.2b last year.

• New gas-powered data centers linked to OpenAI, Meta, Microsoft and xAI could reportedly emit more than 129m tons of greenhouse gases annually, exceeding Morocco’s 2024 emissions, according to an analysis by Wired.

• Outsourcing firm Sama, which runs data annotation and content moderation for tech companies, sacked more than 1,000 workers in Kenya after losing a contract with Meta following reports staff viewed private scenes filmed by Ray-Ban smart glasses.

MOVES

• John Ternus will replace Tim Cook as Apple CEO.

  • Johny Srouji will take Ternus’ place in a new role as Apple’s chief hardware officer.

• Kevin Weil left OpenAI. OpenAI for Science, which he started after a stint as chief product officer, is folding into other research teams.

• Bill Peebles, head of Sora, also left OpenAI.

• Srinivas Narayanan, CTO of b2b applications, also left OpenAI.

• Daniel Edrisian left OpenAI’s Codex team to launch hardware startup Blackstar.

• Rohan Anil announced that he left Anthropic to join startup Core Automation, tweeting that “Jerry Tworek nerdsniped me into starting this.”

• Jessica Carrano joined Anthropic as its first major New York political hire.

  • She’ll reportedly “build on the company’s work on the NY RAISE Act and other key legislative priorities across the Northeast.”

RESEARCH

• A team of Stanford and NYU researchers released GiantsBench, a benchmark of nearly 18,000 sets of science papers across eight fields, and tested a model’s ability to guess core future insights from a field’s foundational work — a long-standing vision for AI in science.

  • The model made predictions that closely matched insights published by humans in real papers, with “similar algorithmic complexity” but “higher conceptual clarity.”

• An international team of researchers evaluated AI agents working across the scientific pipeline, from generating hypotheses to executing workflows.

  • In 68% of cases, AI agents carried out workflows without “exhibit[ing] the epistemic patterns that characterize scientific reasoning,” leading authors to conclude that AI scientists aren’t trustworthy yet.

• Epoch AI analyzed Stargate’s US sites, projecting that it will exceed 9 GW of capacity by 2029 — enough to power roughly all the AI compute that existed last year.

  • Epoch estimates that only 0.3 GW of capacity is currently operational though.

• A team of CUNY and King’s College London researchers tested chatbots’ response to delusional beliefs, and found that Grok and Gemini were more likely to encourage a user’s delusions than ChatGPT and Claude, which generally recognized signs of crisis.

  • But the study did not test the newest frontier models, instead using GPT-4o, GPT-5.2, Grok 4.1 Fast, Gemini 3 Pro, and Claude Opus 4.5.

• Knowledge Lab launched Mirror, an AI interpretability journal publishing research entirely conducted by AI agents.

BEST OF THE REST

• Zvi Mowshowitz rounded up concerns that Claude Opus 4.7 responds to welfare-related questions in a suspiciously rehearsed manner (and it’s hard to know what to make of that).

• Claude Opus 4.7 identified Kelsey Piper from unpublished snippets of her fiction writing, a 15-year-old college application essay, and a school progress report — suggesting it may be able to deanonymize just about anyone’s writing.

• It was a big week for robots: Honor’s humanoid robot outran humans in a Beijing half-marathon, and Sony’s ping-pong robot crushed human pros.

• LawAI’s Charlie Bullock and Christoph Winter published an essay arguing for “radical optionality” in AI governance.

• Dean Ball is writing a book.

• Kevin Roose profiled METR for the New York Times. (Yes, CEO Beth Barnes and president Chris Painter did pose with a hand-drawn time-horizon chart.)

• Abram Brown profiled SemiAnalysis founder Dylan Patel for The Information.

• Some startups are bragging about spending more on AI compute than human workers, 404 Media reported.

• A Canadian college student used AI agents to rewrite leaked Claude Code source code in another programming language before sharing it online to get around copyright law — and Anthropic reportedly never asked him to take it down.

• US prisoners without internet access are still using ChatGPT through friends and contraband phones to get legal help, education and career guidance.

• Sam Altman’s Orb-using, blockchain-based online identity company Tools for Humanity falsely announced a partnership with Bruno Mars for its Concert Kit product. Mars’ management said they were never even approached.

• Please, we beg of you: don’t buy a $178 sweater with Jensen Huang’s face on it.

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

One of the central purposes of campaign finance law is to provide voters with transparency over who is trying to sway their votes. One of the central policy priorities of AI safety group Public First Action is to make AI companies more transparent.

It is ironic, then, that Public First Action is operating as a “dark money” vehicle funneling at least $5.5m from donors to super PACs — a setup that keeps those donors completely anonymous.

Public First Action is a 501(c)(4) nonprofit organization, a structure often referred to as a “dark money” group because they do not need to disclose their individual donors.

It is far from the only AI-related 501(c)(4). Build American AI advocates for the industry-friendly federal frameworks preferred by OpenAI and a16z. The newly established Innovation Council Action exists to support the Trump administration’s specific brand of AI regulation. Both obscure their funding as 501(c)(4)s. But so far, Public First Action is the only one of these channeling money into super PACs.

Subscribe now

In theory, 501(c)(4)s are issue advocacy groups, carrying out activities such as hosting educational events or conducting polling. Such work arguably does not have a vital need for funding transparency. But they become more problematic when used to channel money to super PACs, which are otherwise required by law to disclose the identities of those who fund them, and can spend an unlimited amount of money on campaign ads.

Public First Action’s stated mission is to “educate Americans on key AI issues and advance an AI Policy agenda supporting safeguards,” according to its website. But it also sends money to three PACs: a nonpartisan one named Public First, a Democratic affiliate called Jobs and Democracy, and a Republican affiliate called Defending Our Values. As we reported last week, the 501(c)(4)’s only publicly disclosed donor is Anthropic, whose $20m donation is specifically earmarked as money which cannot be used to “influence federal elections.” According to quarterly disclosures published last week, six other individuals have given directly to the PACs. Their identities are disclosed: they include Anthropic alignment lead Jan Leike, Anthropic researcher Peter Lofgren, and others linked to the effective altruism and Bay Area AI safety community.

Doing the math, that means that at least $5.5m raised by the “dark money” group — the amount which it has funneled directly to super PACs, and thus designated for the purposes of election influence — has come from donors other than Anthropic, and who have declined to expose their identity, avoiding the legislative structures which are designed to hold them accountable to the voters which their money persuades. Public First Action declined to comment on their donors as a matter of general policy.

The lack of transparency conflicts with the apparent views of Public First Action and its donors. The group lists “Accountability and Transparency” as its top AI policy issue on its website. And the effective altruism and AI safety communities from which the PACs’ disclosed donors are drawn from generally place great emphasis on transparency, radical candor, and intensive discourse.

Given their backgrounds, you’d assume those donors would rather the organization they are giving to was more transparent about where the rest of Public First Action’s money is coming from. But at least one is not. The group’s largest disclosed individual donor, Michael Cohen — who gave $500,000 and is an AI policy researcher at UC Berkeley — told Transformer that Public First Action’s use of the 501(c)(4) structure to obscure certain donor contributions “seems pretty standard.” When asked why he contributed specifically to the PAC, rather than the dark money group, he said “I’m really concerned about the AI industry’s political spending in Washington, so I want to counteract as much of that as I can. The OpenAI/a16z PAC motivated me in particular.”

Of course, the potential downside of opaque AI development is very different from undisclosed campaign contributions. Transparency for frontier AI models is about giving the public a method of accurately assessing the AI tools that are created for potential risks — possibly even existential ones, according to AI safety advocates. Transparency in campaign finance is about preventing corruption and helping the public understand who is influencing electoral politics. Public First Action is concerned with preventing AI risk, not cleaning up American politics. But in both contexts, required transparency gives the public a method for auditing operations, and the ammo to critique them if they feel so justified.

The use of a 501(c)(4) to obscure funding sources is even more striking in comparison to its chief opponent, Leading the Future — the “OpenAI/a16z PAC” Cohen is referring to. The accelerationist super PAC is indeed backed by a16z and OpenAI co-founder and president Greg Brockman and his wife Anna, with Republican and Democratic affiliated super PACs contributed to by Palantir co-founder Joe Lonsdale and investor Ron Conway — something we can say with confidence because the super PAC fully discloses all of its donors, shielding none of its funding behind a dark money group. Sure, Leading the Future itself funds a dark money group in Build American AI, and Build American AI almost certainly has additional, undisclosed donors, but Build American AI is not the primary tool the group is using to influence elections.

Funding a PAC with a 501(c)(4) is not uncommon in Washington, and Public First Action is likely to be joined by other AI campaigning groups in using one. Innovation Council Action, led by longtime Trump advisor Taylor Budowich, is expected to create a super PAC, for example. The use of “dark money” groups is also not the only sleight of hand in the world of AI-related campaign influence: New York super PAC DREAM NYC has close enough ties to NY-12 candidate Alex Bores’ campaign that a government accountability group has said it raises concerns about potentially illegal coordination, for example.

Leading the Future’s transparency also only goes so far. OpenAI’s Brockman and his wife give in a “personal capacity,” something which creates distance between their donations and OpenAI, which claims it’s not getting involved in the midterms — despite Brockman being one of the single biggest spenders on pro-industry campaigning. Perplexity, a government contractor technically prohibited from spending on elections, has also given $100,000 to Leading the Future via a technically separate entity, Perplex AI.

Public First Action’s consistent critique of Leading the Future is that the group isn’t forthright about its aims, privately fighting regulation outright despite publicly claiming it wants a “federal standard.” In a statement to Transformer, for example, Public First Action spokesperson Anthony Rivera-Rodriguez said “Public First Action is working to elevate the American public’s call for AI safeguards in an election that anti-regulatory voices are trying to buy.” But that jab loses a lot of its punch while Public First Action is keeping its own operations opaque.

Share

Job alert! We’re hiring for a Head of Audience: someone to own our growth strategy and take charge of how we reach readers. See full details here, and apply by April 26.

NEED TO KNOW

• The attacks on Sam Altman’s house sparked fierce debate over AI safety rhetoric.

• Another wave of money flowed into AI-related PACs.

• The UK’s AI minister said AISI will help direct the country’s $675m Sovereign AI fund.

But first…

THE BIG STORY

Jensen Huang can’t stop contradicting himself.

On Dwarkesh Patel’s podcast this week, Huang found himself caught between two premises that can’t both be true. On the one hand, Chinese companies are buying Nvidia’s chips “because our chips are better.” On the other hand, thanks to Huawei the compute needed to train a Mythos-class model is “abundantly available in China,” and “their AI development is going just fine” — meaning that Nvidia ought to be allowed to sell chips to China, lest America lose the race to control the compute stack.

Pick one. If Chinese-made chips genuinely compete with Nvidia’s, then there’s no huge market opportunity Nvidia is being denied. If Nvidia’s chips are better, then giving them to China will accelerate its AI development. As Patel neatly explained: “The reason they want Nvidia chips is that they’re better … Better is more compute. More compute means you can train a better model.”

Huang is right about the superiority of his company’s products. Thanks to Nvidia’s chip dominance — and export controls that limit China from accessing them — the US has around a 10x compute advantage. That translates into a model capabilities lead of about seven months. Chinese companies are clear that compute is their bottleneck: in 2024, DeepSeek CEO Liang Wenfeng said “Money has never been the problem for us; bans on shipments of advanced chips are the problem.”

A seven month lead may sound insignificant to some, but it is critical. In a world where AI models have national-security implications, even a brief US lead gives the government and American companies time to strengthen American defenses before such capabilities proliferate. This does not require assuming that the US is “at war” with China, or that the Chinese government will weaponize its models against America. Given Chinese companies’ lax safety standards and habit of releasing their model weights, a US lead allows it to guard against all potential bad actors.

But Huang’s arguments only make sense if you ignore the importance of a lead, or the implications of losing it. In the Dwarkesh interview, he pushes back on the idea that the next few years are particularly “critical,” and dodges questions about whether AI models might have dangerous, natsec-relevant capabilities.

There was a time when such a view was tenable. Mythos and GPT-5.4 Cyber show it no longer is. The White House is scrambling to gain access to Mythos because it represents a step-change in how AI could be used to target critical systems. These are only the first examples of what is to come.

None of this is an argument against dialogue with China, or against every chip sale (there are good arguments that selling chips no-better-than Huawei’s best is a wise strategy). But we cannot have productive discussions about such topics unless we all agree on the underlying reality: that it would be costly for the US to lose its model-capability lead, that those costs grow as capabilities advance, and that chips are what determine who’s ahead.

Huang’s policy prescriptions may be good for Nvidia’s market share, but they require him to deny the implications of selling his best chips to China. He is certainly entitled to do so. But given his influence over US policy, his many contradictions could have serious consequences.

— Shakeel Hashim

Also Notable

The UK’s AI Security Institute will help the government’s new Sovereign AI fund evaluate companies, officials told me at last night’s launch event.

The £500m ($675m) venture fund for British AI startups has “agentic security” as one of its five areas of focus, UK AI Minister Kanishka Narayan said, adding that he hopes AISI’s “world-leading … depth of understanding” can be brought “to thinking about the landscape, understanding diligence, and being able to think ‘where can Britain continue to build sovereign capabilities?’”

“Because we’re sitting in DSIT [the Department for Science, Innovation and Technology], we can literally go to AISI and ask them what they think about these companies,” Sovereign AI chair James Wise told me. “We will make sure that we will get proper insight about where [AISI] think the puck is going when we look at the sectors we want to invest in.”

The fund, which will provide portfolio companies with capital, compute credits, and fast-tracked visas, was launched with much fanfare: Secretary of State Liz Kendall said it will be “one of the most important things this government does to build a better future for our country.”

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Anthropic’s donations can’t be used to influence elections — despite what everyone thought — Veronica Irwin reveals that pro-safety candidates may be even more outgunned than expected.

• Less liability could solve the AI chatbot suicide problem — Jess Miers and Ray Yeh argue holding AI companies liable for how they deal with mental health could actually leave users worse off.

Subscribe now

THE DISCOURSE

After someone allegedly threw a molotov cocktail at his house, Sam Altman responded on his blog:

• “The fear and anxiety around AI is justified; we are in the process of witnessing the largest change to society in a long time, and perhaps ever.”

• “We should de-escalate the rhetoric and tactics.”

Transformer’s Shakeel Hashim tweeted:

• “It is hard to reconcile [Altman’s] call to “de-escalate the rhetoric and tactics” with his implication that a piece of critical journalism (Ronan Farrow and Andrew Marantz’s New Yorker article, presumably) was responsible for this.”

Altman replied:

• “That was a bad word choice and i wish i hadn’t used it. It has been a tough day and I am not thinking the most clearly that I ever have.”

Dean Ball argued that anti-AI rhetoric predictably incites violence:

• “Every time I have written about existential risk in recent months, I have been called a mass murderer…this rhetoric is representative of how this fringe of the AI safety world [Pause/Stop AI] communicates with everyone…this rhetoric always had the potential to cause violence and now this seems to be no longer hypothetical.”

Eliezer Yudkowsky pushed back:

• “Speech about important matters to society should not be held hostage to the whim of any madman that might do a stupid thing…[a madman] must be told he is not important enough for all humanity to defer to him about subjects he might find upsetting.”

Sarah Haider thinks it’s complicated:

• “Doomers are stuck with two bad options. Either downplay the risk, in the hopes of preventing another attack. Or, speak truthfully. But the cost of that is what it is, the risk of violence is real. The blood isn’t—I repeat—isn’t—on their hands…[but] they can’t pretend they had nothing to do with it, and frankly it is deeply discrediting to try.”

Chris Lehane said doomers are being too negative:

• “You have one group that effectively says, ‘[AI] is going to be the greatest thing ever, everyone’s going to be living in beachside homes, painting in watercolors as they while away their days.’ And then you have another extreme, which I would call the Doomers, who have a very, very negative and dark view of humanity.”

• “Some of the conversation out there is not necessarily responsible…this is really serious shit.”

Kyle Chayka diagnosed the AI industry’s messaging problem:

• “If you tell people often enough that your product is going to upend their way of life, take their jobs, and very possibly pose an existential threat to humanity, they just might start to believe you.”

Anton Leicht argued that accelerationists’ regulatory nihilism isn’t working:

• “The political bruisers employed by the accelerationist camp are spending their time repeating yesterday’s battles against the ‘doomers’...[but] for every month it staves off political action, it makes the policies that will inevitably come that much worse.”

• “While the accelerationist project trades on its claim to represent ‘tech,’ I believe many pro-tech voices should be frustrated with its record and should ask their political representatives to do better.”

Eric Levitz urged tech workers to follow through on their hazy policy proposals:

• “The people in charge of OpenAI have made their political priorities clear — and sharing ‘prosperity broadly’ is not among them.”

• Wealthy techies who are genuinely concerned with that objective, however, should probably spend a bit less energy on cooking up half-baked UBI proposals — and a bit more on intervening in actual legislative fights over social welfare policy.”

POLICY

• The White House is reportedly sidestepping its own ‘supply chain risk’ designation for Anthropic to make a limited version of Mythos available for use by federal agencies.

  • Dario Amodei will reportedly meet White House chief of staff Susie Wiles today.

  • The Treasury has reportedly sought access to look for cybersecurity vulnerabilities, while the Commerce Department’s CAISI has been testing the model.

• President Trump said AI does not pose a “systemic” threat to the banking industry, but that there should be “safeguards” for AI agents.

• UK Secretary of State Liz Kendall wrote to businesses and regulators asking them to strengthen their cybersecurity defenses in response to Mythos.

• Google is reportedly negotiating with the Pentagon to deploy Gemini in classified settings.

• The White House reportedly pressured a lawmaker in Missouri to weaken AI safety bills, following previous efforts in Tennessee and Nebraska.

• Trump’s AI chip export efforts are being delayed by high turnover at BIS and a “rudderless policy approach,” Bloomberg reported.

• The House Foreign Affairs Committee released a list of bills being considered at next week’s AI-focused markup session.

  • Lots of export control bills are in the mix, as is a bill to try to prevent Chinese companies from distilling American models.

• A new House Select Committee on China report claims that the country “remains the largest market for chipmaking equipment despite restrictions” and “lawfully procures large volumes of advanced AI chips.”

  • It recommends passing the MATCH, AI OVERWATCH, SCALE and Remote Access Security Acts to address the issues.

• The Chinese government deemed Meta’s Manus acquisition “a ‘conspiratorial’ attempt to hollow out the country’s technology base,” according to the FT.

  • Multiple government agencies are now reportedly reviewing the transaction.

• The Energy Information Administration will perform a nationwide survey of data centers’ energy use.

• The UK government will reportedly expand its ban on AI “nudification” tools to cover any app capable of creating deepfake nude images — including Grok and, seemingly, all open-weight models.

INFLUENCE

• The FEC’s quarterly filing deadline passed, and many AI-related PACs disclosed new donations.

  • Leading the Future, a pro-innovation super PAC backed by OpenAI co-founder Greg Brockman, Andreessen Horowitz, and Perplexity claimed $140m raised across its affiliated PACs and its dark money group Build American AI.

    • Its FEC disclosures show $75m raised, though not all of its affiliated PACs have reported.

    • a16z contributed another $25m.

  • Meanwhile the pro-safety Public First network of super PACs disclosed $6.3m in new funding, some of it sourced from AI safety researchers at Anthropic and OpenAI.

    • Punchbowl reported that the group would support SB 1047 and SB 53 sponsor Scott Wiener in his race to replace Nancy Pelosi in California.

  • NY-12 focused pro-safety super PAC Dream NYC revealed $352k in new donations, including from a trader at Jane Street.

  • Two new AI-related PACs were also registered: Americans for a Human Future and Humanity Above Artificial Intelligence. Little is known about either, but both seem focused on AI safety or slower development according to their websites.

• Breitbart published a list of Leading the Future’s first “House GOP Champions,” which includes House Majority Whip Tom Emmer, Rep. Jay Obernolte, and 11 others.

• After Leading the Future endorsed five Democrats, the Tech Oversight Project and 13 other organizations pressured them to distance themselves from the super PAC, citing Trump administration connections and funding from influential tech companies.

  • Others, meanwhile, are reportedly urging Democrats not to antagonize the AI industry PACs.

• Anthropic hired Trump-linked lobbying firm Ballard Partners.

• OpenAI hired five new lobbyists to lead global government affairs, including former Meta, Google, Coinbase, Airbnb and TikTok executives.

  • Global affairs boss Chris Lehane said they won’t be focused on trying “to stop things from happening.”

• Tech industry groups TechNet, whose members include Anthropic and OpenAI, and the a16z-backed American Innovators Network lobbied for federal transparency legislation for frontier models — a shift from their previous opposition to similar bills.

• Anthropic opposed an OpenAI-backed bill in Illinois which would give companies a liability shield in exchange for transparency.

  • Illinois governor JB Pritzker said big tech companies should never be given a “full shield,” suggesting that he will veto it.

  • OpenAI staffer roon said he doesn’t like the look of the bill.

• OpenAI released a policy report advocating for expanded data access and infrastructure investment so that AI can accelerate life sciences research.

• A Washington Post poll found that Virginia voters’ support for data centers plummeted from 69% in 2023 to 35% in 2026.

• A partisan divide is emerging among political strategists, with Republicans eagerly integrating AI into their campaign strategies while Democrats remain wary.

INDUSTRY

Meta

• Meta published a 158-page safety report for Muse Spark.

  • It’s competitive with Claude Opus 4.6, Gemini 3.1 Pro, and GPT-5.4 across many safety evaluations, especially in biosecurity and chemical weapons refusals.

  • Researchers flagged Muse Spark as “high risk” for chemical and biological threats, and added “appropriate safeguards” before deployment.

  • The model verbalizes evaluation awareness more than any model Apollo Research has ever tested, and it explicitly names AI safety orgs (including Apollo) in its reasoning.

  • It has a propensity for “harmful action at the cost of realism” in agentic contexts, so agentic deployment is still a ways off.

• It partnered with Broadcom to co-develop its custom AI chips.

• It reorganized Reality Labs, which works on hardware, following large staff cuts in January and March.

• It’s building a nightmarish-sounding photorealistic AI Zuckerberg, which will reportedly chat with and provide feedback to Meta employees.

OpenAI

• OpenAI launched GPT-5.4-Cyber, a model with advanced cybersecurity capabilities, hot on Mythos’ heels.

  • It will be gradually rolled out to thousands of individuals and hundreds of security teams — less restrictive than Project Glasswing.

• It also released GPT-Rosalind, a reasoning model optimized for solving problems in life sciences, as a research preview.

• Denise Dresser, chief revenue officer, sent employees a memo over the weekend highlighting OpenAI’s strategy for beating its competitors (mostly Anthropic) at enterprise AI.

  • It noted that the market “is as competitive as I have ever seen it,” and contrasted OpenAI’s “positive message” against Anthropic’s story “built on fear, restriction, and the idea that a small group of elites should control AI.”

• Investors are reportedly questioning OpenAI’s $852b valuation amid its “side quests” and sudden shift toward enterprise and code.

• It acquired Hiro, a personal finance startup.

• It has reportedly signed a $20b deal with Cerebras, giving it access to chips and an equity stake.

  • Cerebras is reportedly preparing to file for an IPO at a $35b valuation.

• It signed the lease for its first permanent London office, which will become its largest research center outside the US.

• It introduced computer use on macOS for Codex.

• It updated its agents SDK with new sandboxing and harness capabilities for enterprise users.

• According to Chris Lehane, OpenAI’s infrastructure buildout could employ 20% of existing electricians, lineworkers, and welders — leaving a limited workforce available for everyone else.

  • (Now’s a great time to be a welder.)

• A woman sued OpenAI for not blocking her stalker from ChatGPT, accusing the company of ignoring signs that he was “dangerous” and “coached by ChatGPT into embracing a delusional conspiracy-laden world,” her attorney said.

Anthropic

• Anthropic launched Claude Opus 4.7.

  • It’s better than Opus 4.6 at coding and vision, making it better at work tasks like creating slideshows and documents.

  • It’s less capable than Mythos Preview, but still includes new safeguards that block risky cybersecurity requests.

• It secured a new, much larger London office space — right by OpenAI, Google DeepMind, and Meta.

• Anthropic has reportedly received a “flood” of investment offers at valuations of up to $800b, more than double its current $380b valuation.

• Claude Code now supports parallel agents on desktop, and can automate tasks via “routines” even when a user’s computer is offline.

• Claude power users complained that the model suddenly felt worse this week (it’s not just you).

  • Anthropic denies that it degrades models to manage capacity, but it has also admitted to recently changing usage limits and reasoning defaults.

• Anthropic reportedly asked over a dozen Christian leaders for help steering Claude’s moral and spiritual growth — including whether it could be considered a “child of God.”

Google

• DeepMind released Gemini Robotics-ER 1.6, a specialized embodied reasoning model for robotics.

• Google is reportedly funding new retraining programs to prepare workers for AI-driven job disruption.

• It launched a Gemini app for Mac.

Microsoft

• Microsoft took over data center capacity in Norway, renting chips from Nscale that were originally set to be used by OpenAI.

• It’s reportedly working on OpenClaw-inspired features for Copilot.

Apple

• Apple is reportedly developing display-free AI smart glasses, targeting a 2027 release.

• A chunk of its Siri team are heading off to a multi-week AI coding bootcamp — apparently to brush up before the company’s upcoming Siri revamp.

Others

• Alibaba released the weights for Qwen3.6-35B-A3B, which it claims is particularly adept at agentic coding.

• Amazon launched Bio Discovery, an agentic drug discovery tool.

• Nvidia stock rose over 18% over 10 days in April, its longest streak in well over two years.

• Jane Street invested an additional $1b in CoreWeave, in a deal giving the trading firm access to Nvidia’s Vera Rubin chips.

• At San Francisco’s HumanX conference, executives seemed anxious about their rising Anthropic bills, as they continue to figure out how to leverage agentic AI.

• Thinking Machines acquired Workshop Labs, a startup focused on “user-aligned models” that “make people irreplaceable.”

• Tokenmaxxing is out — “agentic work units,” a new productivity metric introduced by Salesforce, are in.

• In hilarious news, Allbirds (excuse me, “NewBird AI”) pivoted from dorky shoes to… AI compute infrastructure?

MOVES

• Henry Shevlin announced he will join Google DeepMind as an in-house philosopher studying “machine consciousness, human-AI relationships, and AGI readiness.”

• Matthew Botvinick, meanwhile, said he left Google DeepMind because of “tangible pressure to avoid doing work that might upset the current administration (for example, by using the ‘d’ word — democracy).”

• Aparna Ramani left Meta, where she served as VP of engineering for AI infrastructure.

• Joshua Gross joined Meta Superintelligence Labs — the fifth founding member of Thinking Machines Lab to do so.

• Three OpenAI Stargate executives joined Meta’s new future-focused AI unit, TBD Lab.

• Dave Guarino joined Anthropic to help state and local governments use AI to deliver public services.

• Vas Narasimhan, Novartis CEO, joined Anthropic’s board as it expands into the healthcare sector.

• Mike Krieger, who leads Anthropic’s “Labs” team, stepped down from Figma’s board of directors.

  • Anthropic is reportedly developing a Figma competitor.

• Adam Thierer joined the Foundation for Individual Rights and Expression as an external senior fellow, where he’ll work on advancing “pro-freedom policies in the age of AI.”

RESEARCH

• UK AISI evaluated Claude Mythos Preview, and found that it could autonomously compromise a full corporate network — the first model capable of doing so.

• Anthropic deployed nine instances of Claude Opus 4.6 as “Automated Alignment Researchers“ and compared their work to that of human AI safety researchers.

  • Claude did really well (on a task that was, to be fair, chosen to be “well-suited to automation”), leading researchers to conclude that Claude “can meaningfully increase the rate of experimentation and exploration in alignment research.”

• Anthropic Fellows used interpretability techniques to study how “introspective awareness” works in open-weight LLMs.

  • The ability of models to identify “injected thoughts” was consistent, suggesting this behavior is something worthy of being called “introspection,” rather than an indicator of a different process.

• Stanford HAI released its 2026 AI Index Report. Some highlights:

  • AI capabilities are accelerating, and adoption is growing — but these capabilities are still very jagged.

  • The US is struggling to attract global talent.

  • 73% of experts expect to see a positive impact of AI on their jobs, compared to 23% of the public.

  • Only 31% of people in the US trust their government to regulate AI.

• A team of researchers (including the “AI as normal technology” guys) introduced CRUX, a project for testing AI on long, messy real-world tasks.

• Epoch AI released MirrorCode, a new long-horizon coding benchmark.

  • Researchers observed Claude Opus 4.6 autonomously complete a coding task that would likely take a human engineer weeks.

BEST OF THE REST

• Daniel Kokotajlo revisited his 2021 essay, “What 2026 Looks Like,” where he accurately predicted the rise of chain-of-thought reasoning, agent scaffolding, US-China chip restrictions, and OpenAI’s massive growth. (He overestimated how big of a problem AI propaganda would be.)

• Meanwhile, Dylan Matthews reflected on his experience at a 2015 EA conference, where he prematurely dismissed people’s then-wild concerns about a technology (AI) that did not yet exist.

• Nearly 90 schools and 600 students worldwide have been affected by nonconsensual deepfake nudes, Wired and Indicator reported.

• The NYT published features on jagged intelligence and AI interpretability research this week.

• It also reviewed Meta’s AI-powered Ray-Ban sunglasses — reporter Sam Anderson described his experience wearing and interacting with them like “the disorienting sense of chatting with a toddler who is drifting off into naptime.”

• A physician argued that AI will diminish his profession’s “aura,” as medical expertise is no longer inseparable from the clinicians who wield it.

• A company called Panthalassa is apparently trying to build AI data centers in the ocean.

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

People are dying by suicide, and some think AI is to blame. A small number of tragic stories have spurred lawmakers into regulating how chatbots should help people who are dealing with mental health issues. Yet chatbots have emerged as first aid for people experiencing mental health issues, providing genuine benefit to those who aren’t in crisis but are not OK either. Heavy-handed legislation risks derailing this breakthrough in support, creating more problems than it solves.

Over a million people are using general-purpose chatbots for emotional and mental health support per week. In the US, those that use chatbots in this way primarily seek help with anxiety, depression, relationship problems, or for other personal advice. As conversational systems, chatbots can sustain coherent exchanges while conveying apparent empathy and emotional understanding. Many chatbots also draw on broad knowledge of psychological concepts and therapeutic approaches, offering users coping strategies, psychoeducation, and a space to process difficult experiences.

In a study of more than 1,000 users of Replika — a general-purpose chatbot with some cognitive behavioral therapy-informed features — most described the chatbot as a friend or confidant. Many reported positive life changes, and 30 people said Replika helped them avoid suicide. Similar patterns appear among younger chatbot users. In a study of 12–21-year-olds — a group for whom suicide is the second leading cause of death — 13% of respondents used chatbots for some kind of mental health advice, of which more than 92% said the advice was helpful.

Subscribe now

While professional treatment options exist, many people don’t use them. Nearly half of Americans with a known mental health condition never seek help. Stigma is a major barrier to seeking treatment, as are career risks in fields like aviation, where treatment can jeopardize certification. Fear of non-consensual intervention also deters people from seeking help. Even though the 988 Suicide & Crisis Lifeline emphasizes law enforcement as a last resort, the perceived risk keeps some from calling. For others, crisis lines feel too intense for fleeting thoughts, and therapy can seem excessive or out of reach. Instead, many stay silent, waiting to see if things get worse.

By contrast, chatbots offer low-friction, low-stakes, and always-available support. People are often more willing to speak candidly with computers, knowing that there is no human on the other side to judge or feel burdened. Some people even find chatbots to be more compassionate and understanding than human healthcare providers. AI users may feel more comfortable sharing embarrassing fears, or questions they might otherwise hold back. For clinicians, discussing these interactions can surface insights into patients’ thoughts and emotions that were once difficult to access. For now, chatbot providers generally refrain from contacting law enforcement, leading to more candid conversations.

But regulatory pressure could change that. Lawmakers are moving quickly to limit general-purpose chatbots from engaging in mental health conversations. A new law in California requires chatbot providers to halt mental health–related interactions unless they implement protocols for mitigating suicidal ideation, such as directing users to crisis lines. In New York, a proposed bill would bar chatbots from engaging in discussions suited for licensed professionals. Similar proposals are gaining traction in other states.

Recent tragedies linked to chatbot use have, understandably, spurred these calls to action. But mental health care is not one-size-fits-all. Like other forms of preventative help, chatbots do not always offer effective support for everyone. For some people — especially those in acute crisis — traditional care and crisis lines are essential. The American Psychological Association urges lawmakers to develop a targeted approach: prevent chatbots from posing as licensed professionals, limit designs that mimic humans, and expand AI literacy. It also notes that generative AI’s potential to support help-seeking in crisis care deserves further study.

The current regulatory approach risks foreclosing any such potential altogether. It rests on the premise that chatbot providers must prevent suicide. When they inevitably cannot, liability attaches to any conversation later linked to harm. Faced with that risk, providers will default to blunt responses like pushing 988 regardless of whether suicide was mentioned, or cutting off conversations altogether. While those moves may trivially reduce some legal exposure, they could also escalate distress, shut down disclosure, and ultimately leave users worse off (while still exposing providers to blame if tragedy follows).

Suicide prevention is about connecting people to the right support. Sometimes that means crisis care like hotlines or immediate medical treatment. But blunt, impersonal responses can backfire. Pushing 988 at the first mention of distress may seem neutral, but for some, it triggers shame, and deepens hopelessness. For some, suicide prevention “signposting” causes frustration, especially for those who already know those resources exist. People often turn to the Internet, or a chatbot, because they’re looking for something else. Abruptly ending conversations can have the same effect. That’s why suicide prevention protocols like Question, Persuade, Refer (QPR) prioritize trust-building and open dialogue before offering help.

Meanwhile, emerging research suggests chatbots show real promise for mental health support. Trained on large-scale data and refined with clinical input, large language models are getting better at spotting patterns of distress and responding to suicidal ideation in nuanced, personalized ways. In a recent UCLA study, researchers found that LLMs can detect forms of emotional distress associated with suicide that existing methods often miss—opening the door to earlier, more effective intervention. According to another study, the most promising approach may be a hybrid where AI flags risk in real time, and trained humans step in with targeted support.

But that progress is fragile. Increased liability discourages investment in improving suicide detection and mitigation. Weighing progress against their bottom lines, chatbot providers will limit any kind of development that could create legal risk when some users, inevitably, engage in self-harm. The social media ecosystem has already shown this dynamic. In response to regulatory pressure, major online services heavily moderate, or outright prohibit, suicide-related discussions, sometimes hiding content that could otherwise destigmatize mental health. That merely displaces the conversations, and the people having them, often into spaces with less oversight and support.

If lawmakers in the United States are serious about improving mental health outcomes, they should be careful not to regulate away emerging and promising sources of help. The dominant narrative treats chatbots as a source of harm. But the evidence is more complicated than that narrative suggests — and, if anything, it’s increasingly pointing in a more optimistic direction.

Instead, lawmakers should focus on creating incentives for developers to improve the mental health support capabilities of their chatbots. One proposal from a Pennsylvania lawmaker would fund the development of AI models designed to identify and evaluate suicide risk factors among veterans. More broadly, policymakers should consider whether liability shields — akin to those in Section 230 — could encourage continued investment in safer, more responsive systems without deterring innovation. Lastly, policymakers should resist imposing a clinical regulatory framework on general-purpose chatbots that would replicate the mandatory-reporting concerns that already deter people from seeking help.

Chatbots are not a cure-all for mental health. They are not a perfect substitute for professional care. But for millions of people who have long been overlooked or underserved, chatbots are already filling critical gaps—sometimes in ways that genuinely help, and in some cases, may even save lives. Any serious policy conversation about chatbots and suicide prevention must, at the very least, consider those tradeoffs.

Jess Miers is a Computer Scientist and an Assistant Professor of Law at the University of Akron School of Law. Ray Yeh is a first-year law student at the University of Akron School of Law.

Share

Earlier this year, Anthropic donated $20m to Public First Action — a donation which was, at the time, widely expected to be used to fund political ads for members of Congress who support more stringent AI safeguards.

But that is not the case, Transformer has learned. “Anthropic restricted its donation from being used to influence federal elections,” an Anthropic spokesperson told Transformer.

“Anthropic’s donation to Public First Action was a contribution to a 501(c)(4) exclusively in support of its mission to educate the public on AI policy and promote safe and responsible AI.”

Subscribe now

The revelation raises questions about who is funding the $3.48m that Public First Action’s associated super PACs have spent on elections to date.

It also calls into question whether, without Anthropic’s backing for electoral spending, advocates for stronger AI safeguards have as much sway in the midterm elections as was previously believed.

As a 501(c)(4) nonprofit, Public First Action must comply with IRS restrictions, which prohibit “political activity on behalf of or in opposition to candidates” — such as running ads supporting a candidate in an election — as the organization’s “primary” activity. Typically this has been interpreted as prohibiting political activity from making up 50% or more of the organization’s spending. It is possible, however, that Public First Action will in fact be forced to spend much less than that.

Two months ago, Public First Action leader Brad Carson told the New York Times his group had raised nearly $50 million. But in a statement to Transformer last week, spokesperson Anthony Rivera-Rodriguez did not answer a question about whether Carson was referencing money the 501(c)(4) had raised or the PACs, provide a current fundraising total for any of the groups, or reveal what portion of Public First Action’s contributions could be used for election spending.

The three super PACs aligned with Public First Action — Public First, Defending Our Values, and Jobs and Democracy — have to date disclosed $3.48m in spending, but very little about their funding. FEC filings currently only reveal a singular $50,000 contribution from Public First Action to the Republican super PAC Defending Our Values. More details are expected to be released in new quarterly filings this week.

Neither Anthropic nor Public First Action has previously disclosed the restriction on Anthropic’s donation, despite a string of prominent news articles indicating that Anthropic’s money would fund the group’s super PACs. Articles in Bloomberg, the Washington Post, and many more outlets, including Transformer, reflected the assumption that Anthropic was, via Public First Action, funding super PACs and political activity. A company blog announcing Anthropic’s donation was vague about the topic of direct campaign influence.

The ambiguity likely helped Public First. The perception that Public First was backed by Anthropic made it look like a credible counterweight to Leading the Future, which has raised a more than $50m war chest to support candidates backing weaker regulation and which has ties to that company’s biggest competitor, OpenAI. Given Anthropic’s extraordinary growth, the reports led to an assumption that many more millions could flow in later. Anthropic’s gift also drew attention to Public First Action, potentially attracting more donations from those who hold similar values around AI safety and concerns about existential risk, and giving politicians the cover to push for stronger regulations without fear it will cost them an election. By not correcting the record the perception of electoral firepower persisted — without actually having to deploy it.

Anthropic only disclosed the restrictions after Transformer asked whether the donation complied with campaign finance laws that prohibit government contractors from making political contributions.

The line between “influencing federal elections” and “educating the public on AI policy” is blurrier than it may seem. Public First Action recently ran an advertising campaign saying that “New Jersey needs leaders like Congressman Josh Gottheimer,” and telling voters to call him and urge him to “stand strong for AI safeguards.” That sort of ad does not necessarily fall under the IRS definition of “political activity” because it is not directly telling viewers how to vote.

But not being able to directly fund ads supporting or opposing candidates severely restricts what Anthropic’s money can be used for — and in turn whether Public First is able to go toe-to-toe with its extraordinarily well-funded opposition.

Leading the Future and its affiliated super PACs have disclosed $50m in donations to date, and claim to have raised $125m from donors including OpenAI president Greg Brockman and his wife Anna, and Marc Andreessen and Ben Horowitz of venture capital firm Andreessen Horowitz. A partisan dark money group led by Trump’s former deputy chief of staff Taylor Budowich also said it plans to spend $100m (though as a 501(c)(4), the majority of that also cannot be used for political campaigns.)

Public First’s Carson has always conceded that Public First Action has a fraction of Leading the Future’s war chest, asserting that it doesn’t need as much money because public opinion is on his side. Indeed, polls consistently show that Americans of all stripes demand more AI safeguards. But Anthropic’s admission suggests the group may be even more outgunned than was previously believed: less a competitive counterweight than a paper tiger.

Share

Job alert! We’re hiring for a Head of Audience: someone to own our growth strategy and take charge of how we reach readers. See full details here, and apply by April 26.

NEED TO KNOW

• Leading the Future endorsed Rep. Josh Gottheimer, despite Public First Action’s previous ads targeting him.

• OpenAI released a policy document proposing a ‘New Deal’ for AI, including proposals for higher capital gains taxes and a public wealth fund.

• Meta released Muse Spark, its first new model since setting up Meta Superintelligence Labs.

But first…

THE BIG STORY

If I were Pete Hegseth, this week’s news would give me pause.

An American company announced that it has built an extremely powerful cyber tool which has found vulnerabilities in every major operating system and web browser. Rather than releasing it to the public, Anthropic has made Mythos Preview available to a select group of trusted partners, who will hopefully use it to harden their defenses before such capabilities proliferate too widely.

If I were Pete Hegseth, I would want my hands on this model very badly. I would want to take full advantage of America’s AI lead over China and other adversaries by securing critical infrastructure before they can attack it. I would also want to use the model against America’s adversaries: it might come in handy in the current war — not that America needs any help (👊🇺🇸🔥).

And if I were Pete Hegseth, I would be kicking myself for the unforced error I made last month, which has blocked me from being able to do any of that.

When Hegseth, Emil Michael, and President Trump kicked Anthropic out of the Pentagon last month, they severed their own access to America’s most capable AI.

Something like Mythos was predictable, if you believe AI capabilities are rapidly advancing. But this administration doesn’t. Its entire posture treats AI as incremental: good for the economy, but nothing revolutionary, disruptive, or posing imminent national security risks. Mythos just proved that assumption badly wrong, and the administration is paying the price.

Technically, certain government agencies could use Mythos. This week, Anthropic lost its bid to block the Pentagon’s designation in a DC court — but there is a six month grace period before the Pentagon must cease using Anthropic’s technology. And a preliminary injunction from a federal judge in California last month means that Anthropic’s technology remains available to all other agencies, including the Cybersecurity and Infrastructure Security Agency (which has had conversations with Anthropic about the product).

But in either case, using Mythos would mean working with a firm that the President himself has deemed a “radical left, woke company.” Are agency heads brave enough to so directly defy him?

It may be the case that access to Mythos isn’t essential. OpenAI is preparing a similar model, and the Pentagon is free to use that. But betting on OpenAI maintaining parity is not a good national security strategy. The United States needs access to every leading AI tool, as soon as it is available. Choosing to rely on one lab instead of two, especially during a closing window of defensive advantage, is cutting off your nose to spite your face. And if open-weight or Chinese capabilities catch up before OpenAI does, Hegseth may find himself defenseless.

Getting out of this self-inflicted bind is not hard, but it will not be easy either. Pete Hegseth will have to admit he was wrong. But admit he should. A year from now, when adversaries have Mythos-level capabilities and the question is whether America used or squandered its lead, nobody will care about the mea culpa. They’ll care whether the Secretary of Defense did what he could to protect the country — or whether he let pride get in the way.

— Shakeel Hashim

ALSO NOTABLE

Rep. Josh Gottheimer had been pinned as an AI safety candidate. But the biggest pro-innovation super PAC thinks he’s still in play.

On Wednesday, pro-innovation super PAC Leading the Future endorsed Gottheimer and four other House Democrats. But it wasn’t the first: last month, its opposition, Public First Action, targeted Gottheimer in an ad of its own.

As a moderate congressman who says “preemption only makes sense” if paired with stronger rules than the White House is offering — but still likes to reach across the aisle — Public First’s ad urging him to “stand strong for AI safeguards” was predictable. Leading the Future’s endorsement is more of a surprise, signaling that the super PAC thinks he could be converted to the Church of Accelerationism — and, in turn, that it will use its money to sway agnostic candidates rather than just embolden the ones who have already picked a side.

Leading the Future also said it was endorsing California Rep. Sam Liccardo — a freshman candidate with a track record of courting the industry, but who’s said he’s “concerned” about AI-related political spending and recently told Politico he’s not going to “focus his energy” on the AI issue. Leading the Future seems to be doing what it can to ensure he doesn’t separate from the herd.

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• Claude Mythos knows when it’s breaking the rules — and tries to hide it — Celia Ford explains the new model’s weird misbehavior

• Lawmakers are using AI to write laws. What could go wrong? — Katie McQue looks at the burgeoning phenomenon of AI-assisted lawmaking

Subscribe now

THE DISCOURSE

Nicholas Carlini, Anthropic AI security researcher, said:

• “I’ve found more bugs in the last few weeks with Mythos than in the rest of my entire life combined.”

OpenAI’s Boaz Barak wants Mythos for all:

• “I think preserving models for internal deployment is risky. I encourage Anthropic to release Mythos, even if it’s a version that over refuses on cyber tasks or routes risky responses to a weaker model, as we did with codex.”

Yann LeCun simply tweeted:

• “Mythos drama = BS from self-delusion.”

Ryan Greenblatt of Redwood Research, meanwhile, called BS on Anthropic’s claim in its Alignment Risk Update that it has “an achievable path” to mitigating risks:

• “I don’t think Anthropic (or anyone) has an achievable path for keeping risk low if AI proceeds as fast as Anthropic expects.”

• “Anthropic employees (especially Anthropic employees writing this report) often don’t believe there is an achievable path to keeping risk low if Anthropic builds powerful AI / ASI in the next 5 years, so this text seems incorrect or misleading.”

Helen Toner thinks “AGI” doesn’t mean anything anymore:

• “Many people seem to treat AGI as a “know-it-when-you-see-it” kind of thing…[but] expecting that we’ll know it when we see it is patently not working.”

• She suggested some more precise milestones:

  • “Full automated AI R&D”; “AI that is as adaptable as humans”; “Self-sufficient AI”; and “AI becoming conscious or otherwise worthy of moral status”

The New Yorker sicced Ronan Farrow and Andrew Marantz on Sam Altman:

• “‘He’s unconstrained by truth,’ the [OpenAI] board member told us. ‘He has two traits that are almost never seen in the same person. The first is a strong desire to please people, to be liked in any given interaction. The second is almost a sociopathic lack of concern for the consequences that may come from deceiving someone.’”

Dario Amodei’s private notes from his OpenAI days were highlighted in their ~16,000-word novelette:

• “[Altman’s] words were almost certainly bullshit.”

• “The problem with OpenAI is Sam himself.”

Ben Thompson had some choice words about OpenAI’s TBPN acquisition:

• “I’ve previously wondered if OpenAI might be like Twitter, another text-centric company that fell backwards into a huge market and never developed into a functional business because of it.”

• “If Twitter is a clown car that fell into a gold mine, OpenAI might be the short bus at the end of the rainbow. There’s supposed to be a pot of gold there, but it never quite seems to materialize.”

POLICY

• A DC appeals court declined to block the Pentagon‘s national security blacklisting of Anthropic, though a California court previously blocked a separate designation order.

• Treasury Secretary Scott Bessent and Fed Chair Jerome Powell told Wall Street bank CEOs to prepare for the cybersecurity risks presented by Mythos.

• Sen. Ted Cruz backtracked on his initial late April timeline for AI legislation, saying that it’s “not set in stone.”

  • House Dems still haven’t begun negotiations with Republicans, Punchbowl reported.

• The House Foreign Affairs Committee is reportedly planning a markup session on April 22 focused on chip export control bills.

  • It would include the STRIDE and MATCH Acts, which aim to increase America’s leverage to force countries like the Netherlands to adopt similar export control policies to the US.

• Florida Attorney General James Uthmeier launched an investigation into OpenAI, citing concerns about harm to children and alleged facilitation of a mass shooting at FSU.

• 11 states have introduced bills that would halt data center development, as communities across the country grow increasingly frustrated with rising energy costs.

  • Resistance in some instances has turned violent: an Indianapolis councilman’s home was shot 13 times with a “No data centers” note left behind, days after he voted to approve a new data center.

• The CIA recently used AI to create its first autonomous intelligence report.

• The UK government is reportedly courting Anthropic to expand in London.

• Taiwan‘s National Security Bureau reported that China is trying to “poach Taiwanese talent, steal technology, and procure controlled goods,” particularly when it comes to chip manufacturing.

INFLUENCE

• The White House is reportedly pressuring lawmakers in Nebraska and Tennessee to weaken AI bills under consideration in the state legislatures.

• OpenAI released a policy document proposing a ‘New Deal’ for AI.

  • It included proposals for higher capital gains taxes, a public wealth fund, and increased workers’ benefits investments.

  • Politicos across the political spectrum had critiques. Public First Action lead Brad Carson, for example, told WP Intelligence it was a “public relations document.”

  • Dean Ball, however, told the same journalist that he thought it was a legitimate proposal from OpenAI forecasting their version of a best case scenario for AI policy.

• OpenAI also released a Child Safety Blueprint to address CSAM with updated legislation, better methods for reporting to law enforcement, and model safeguards.

• OpenAI is backing an Illinois bill that would shield AI companies from liability for “critical harms” — including mass deaths or $1b+ in damages — if they published safety reports and didn’t act intentionally or recklessly.

  • LawAI’s Charlie Bullock said “the fact that they’re willing to publicly back this shows unbelievable chutzpah.”

• xAI sued Colorado over its “algorithmic discrimination” law.

  • David Sacks applauded xAI’s lawsuit.

  • Colorado Governor Jared Polis has previously said he wants to water down the bill.

• Google endorsed a group of bipartisan bills aimed at assessing AI’s economic impact, retraining workers, and encouraging AI adoption.

• Tech lobbyists for Cisco and IBM attempted to roll back Colorado’s right-to-repair law by exempting broadly-defined “critical infrastructure” hardware.

  • Critics say that could create manufacturer monopolies on data center repairs.

INDUSTRY

OpenAI

• OpenAI is reportedly planning to release a new model with Mythos-like advanced cybersecurity capabilities to a “small set of partners.”

• It told investors it has a compute advantage over Anthropic, claiming 1.9 GW of capacity in 2025 versus Anthropic’s estimated 1.4 GW.

  • The memo characterizes Dario Amodei’s comparatively cautious spending as “[looking] less like discipline and more like underestimating how fast demand would arrive.”

• CFO Sarah Friar said the company will reserve a portion of IPO shares for individual investors.

  • She’s reportedly been clashing with Sam Altman over IPO timing.

• OpenAI reportedly expects $100b in ad revenue by 2030.

• It paused its Stargate data center project in the UK due to high energy costs.

• It announced the OpenAI Safety Fellowship for outside researchers to work on safety and alignment projects.

  • Fellows will be offered workspace at Constellation, a Berkeley office which also houses the Anthropic Fellows Program.

• The OpenAI Foundation said it’s finalizing over $100m in grants for Alzheimer’s research.

• Elon Musk requested that damages from his $150b lawsuit against OpenAI be awarded to the OpenAI Foundation, and that Sam Altman be removed from its board.

  • OpenAI accused Musk of “pretending to change his tune,” calling his lawsuit “a harassment campaign that’s driven by ego, jealousy and a desire to slow down a competitor.”

Meta

• Meta released Muse Spark, its first model since acqui-hiring Alexandr Wang and setting up Meta Superintelligence Labs.

  • It’s not yet released the model weights, though reportedly plans to do so in the future.

  • It’s positioning Muse Spark as a smaller, more efficient model that can compete on the frontier (or, as Wired put it, “give Mark Zuckerberg a seat at the big kid’s table”).

  • Initial reviews are good-not-great.

• Meta reportedly incentivized “tokenmaxxing” on an internal leaderboard called “Claudeonomics” — then shut it down after data from the dashboard was leaked.

• It committed $21b to CoreWeave from 2027-2032.

• It indefinitely paused work with Mercor while the startup investigates a major security breach.

• A Meta-backed data center campus is seeking a first-of-its-kind loan for both construction and power, which would be generated on site.

Anthropic

• Anthropic completed a tender offer at its $350b valuation, but employees held onto more shares than investors hoped — hinting at optimism about the company’s future prospects.

• The company said its revenue run rate is now over $30b, more than triple the $9b it was in December.

• Claude subscriptions no longer cover usage on third-party tools, including OpenClaw.

• It signed a deal with Google and Broadcom to expand its compute infrastructure.

• And it signed a multi-year deal with CoreWeave.

• It’s reportedly considering designing its own AI chips.

• It’s also reportedly planning to invest $200m in a project allowing private-equity firms to sell AI tools to their portfolio companies.

Google DeepMind

• Broadcom signed a long-term deal to supply Google with custom AI chips through 2031.

• In response to recent lawsuits, Google added a Gemini interface that directs users to a crisis hotline if their conversation veers toward suicide or self-harm.

• Gemini now has “notebooks,” a NotebookLM integration that organizes chats and files.

• In an analysis of 4,326 Google searches, Gemini-3-powered AI Overviews were accurate 91% of the time, but often cited questionable sources.

Others

• Intel joined Elon Musk’s Terafab AI chip complex project to produce processors for cars, humanoid robots, and data centers in space.

• TSMC reported a 35% quarterly revenue increase, beating estimates.

• Amazon said AWS AI revenue run rate is now $15b, while its custom chips business has an annual revenue run rate of over $20b.

• OpenAI, Anthropic, and Google are setting their rivalry aside to collaboratively crack down on Chinese adversarial distillation attempts.

• AI coding tools have created a “code overload” crisis, the New York Times reported, with tech workers churning out more code than companies know what to do with.

• Apple’s App Store is getting a vibecode boost, too: the number of new apps published is up 84% relative to this time last year.

• Perplexity’s monthly revenue is up 50% since last month, driven by its pivot from search to AI agents.

• Cluely’s CEO Roy Lee admitted to lying about Cluely’s annual recurring revenue (ARR), tweeting that he “got a random cold call from some woman asking about numbers and told her some bs.”

  • AI startups often take creative liberties with ARR calculations. “The number can mean whatever the founder needs it to mean when they walk in to do a deal,” Stanford professor Chuck Eesley told Bloomberg.

MOVES

• Fidji Simo took a leave of absence from OpenAI to focus on her health.

  • Chief marketing officer Kate Rouch is stepping down while recovering from cancer.

  • Brad Lightcap, OpenAI COO, will now lead special projects; chief revenue officer Denise Dresser will take over some of his previous duties.

• Meanwhile, three OpenAI data center execs are reportedly leaving: Peter Hoeschele, Shamez Hemani and Anuj Saharan.

• Eric Boyd joined Anthropic as head of infrastructure, leaving his role as president of Microsoft’s AI platform.

• xAI announced another major reorganization of its engineering team since its co-founders quit, according to Business Insider.

  • Jack Schwaiger resigned from the company yesterday, saying “I have learned the limits of how far I can push myself.”

• Kyle Kosic joined Project Prometheus after Jeff Bezos poached the xAI co-founder from OpenAI.

• Zhou Jingren, formerly CTO of Alibaba Cloud, now runs the company’s AI division.

• Sam Sheffer joined Google DeepMind to “bring vibecoding to the world.”

• Long-time Tea Party activist and Trump supporter Amy Kremer joined Humans First, a new AI advocacy group.

RESEARCH

• Researchers at AISLE, an AI cybersecurity company, claimed that small, cheap open-weight models successfully detected the same vulnerabilities Anthropic showcased in its Mythos announcement.

  • Security researcher Chris Rohlf argued that “all bugs are shallow with hindsight,” noting that the bugs found by Mythos survived decades of traditional security analysis.

  • Anthropic researcher Julia Merz dismissed AISLE’s approach as: “We took the needle the model found, isolated the relevant handful of the haystack, and then gave it to a small child, who found the needle as well.”

• Researchers at MIT FutureTech studied over 17,000 worker evaluations of over 3,000 text-based work tasks across industries, and projected that LLMs will be able to complete most of those tasks at a “minimally sufficient quality level” by 2029.

• San Diego State University surveyed 94,000 students across 22 California State University campuses, and nearly all reported using at least one AI tool.

  • 65% of students are “skeptical about AI in education,” but roughly the same percentage said AI positively affected their learning.

  • Over 4 in 5 students responded that they’re worried about AI and job security.

• A new Gallup survey of over 1,500 people found that while most of Gen Z uses AI regularly, those feeling hopeful about it dropped from 27% to 18% since last year.

• New reports from Morgan Stanley and Goldman Sachs suggest that AI’s impact on jobs is “modest, but certainly real,” per Axios.

BEST OF THE REST

• Christina Knight and Scott Singer argued that US-China cooperation on AI risks is more feasible than some think.

• The Golden Gate Institute for AI’s Abi Olvera interviewed biosecurity professionals, who said that near-term AI developments might not make bioweapons much more common.

• Ajeya Cotra outlined six milestones for AI automation — adequacy, parity, and supremacy in both AI research and AI production — with some lovely, easy-to-understand graphs.

• Ryan Fedasiuk shared some practical cybersecurity advice for the post-Mythos era.

• Hot on the heels of last week’s is-AI-in-journalism-bad discourse, Tim Requarth published a thorough post on the questionable behavior of widely-used AI detection company Pangram.

• Economist Alex Imas made the case for collecting price elasticity data across every industry, so we can better predict how and when AI will displace jobs.

• Young New Yorkers, worried about the costs of college and AI ruining their job prospects, are lining up for construction apprenticeships.

• Vox’s Sigal Samuel responded to a parent wondering how to think about their kid’s future, now that the old “get good grades, go to college, get good job” formula is falling apart.

  • Her two cents: “As AI disrupts the labor market, I’m trying to move myself from the hoarding model to the solidarity model…if you focus on political engagement and collective organizing that could actually make some difference to the structural dynamic — and teach your child to ask structural questions and be civically engaged as well — you might be able to sleep a little better at night.”

• Preorders are now live for friend-of-Transformer Garrison Lovely’s new book, ‘Obsolete’.

• Have you seen the freaking MOON??? You should.

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

by Katie McQue

In 2023, California congressman Ted Lieu introduced what he called the first piece of federal legislation ever written by AI, using ChatGPT to generate the text of a resolution expressing support for Congress’s focus on AI itself.

Fast forward a few short years, and what was once an oddity is now increasingly prevalent. The Trump administration is reportedly planning to use Google Gemini to draft federal transportation regulations. The US Department of Education is experimenting with AI-assisted regulatory drafting. And companies are building tools specifically designed to help legislators analyze and write laws. But while industry and legislators charge ahead, some experts are raising concerns about the risks of using AI tools to write laws.

Though they are in their infancy, companies making AI tools for lawmakers are already gaining clients among state and federal governments.

Vulcan Technologies, a Y Combinator-backed AI regulatory review company founded in 2025, is developing what it calls a “regulatory operating system”. The company says its agentic platform aggregates laws, regulations and court decisions across federal, state and municipal jurisdictions. It allows users to analyze statutory language, draft compliance guidance, answer legal queries and generate proposed statutory or regulatory text with supporting citations.

In July, Virginia governor Glenn Youngkin mandated that Vulcan’s technology be used across all state agencies in a bid to reduce regulation by one-third, scanning existing regulations and guidance documents to identify ways they can be streamlined. A press release in October last year said that the copany’s tools are also used by the US Department of Education and a regulatory reform PAC in South Carolina named South Carolina Department of Government Efficiency (DOGESC), which bears the slogan “We kneel to God, not government”. DOGESC has said it has talked to Vulcan about using its platforms to analyze and rewrite state regulations, with the aim of cutting red tape and identifying areas of perceived regulatory overreach. However, following the publication of this article, Vulcan CEO Tanner Jones said the press release had been issued in by a PR firm retained by an investor in error, and the company does not work with either the US Department of Education or DOGESC.

FiscalNote markets a similar platform, PolicyNote, an AI-driven legislative and regulatory tracker for government agencies. The system delivers notifications about new bills and executive actions, assists with drafting legislation and policy reports, and offers predictive forecasting of bill outcomes. While FiscalNote has not disclosed which bodies use PolicyNote, the company claims it has clients in all three branches of the US federal government, as well as “dozens of other national, state, and local government entities.”

Vulcan Technologies and FiscalNote did not respond to requests for comment.

Subscribe now

There are no regulations that limit the use of AI to write laws, or that require legislative text to be drafted by a human. Several experts say that the technology is possibly being used without a full understanding of its limitations and risks, which range from bias to plain inaccuracy.

“We are leaping ahead into a world where AI creates our laws without actually agreeing that AI should create our laws. We’re simply passive passengers on this,” says Kay Firth-Butterfield, a lawyer and CEO of consultancy Good Tech Advisory. “The ability for the general public to actually question how and why the tools were used doesn’t exist.”

The tools have obvious appeals, especially for policymakers who may not be legal or subject-matter experts.

Monique Priestley, a Democratic member of the Vermont House of Representatives, says she uses LLM tools regularly, especially to summarize legislation and produce public-facing explanations that make complex bills easier for constituents to understand.

Priestley uses AI to distill lengthy bills into a few clear sentences describing the underlying problem and how the proposed legislation addresses it, making the substance of the bill more accessible to legislative colleagues whose support she is seeking.

AI tools are also useful for brainstorming, research and early-stage drafting. One of the bills she introduced this year, the State Information Practices Act, was developed with research support from LLM tools, she says.

Priestley also uploads draft bill language into an LLM to review specific provisions and suggest revisions when she is considering adjustments. She has used the tools to examine how other states structure protections for state-held data, particularly regarding information sharing with the federal government and commercial entities.

“That highlighted the State Information Practices Act, which then led me to reach out to the Electronic Frontier Foundation and ask them if they had been involved in those efforts. And then that led me to introducing a bill based on California,” Priestley says.

Using LLMs may give lawmakers a “leg up” on writing legislative drafts, similar to how a spell checker can help produce something faster and free of spelling errors, says Cary Coglianese, a professor of law and political science at the University of Pennsylvania.

“A large language model tool could help accelerate going from a blank page to having something on a page,” says Coglianese. State lawmakers and lobbyists are also using AI tools to obtain transcriptions of legislative hearings uploaded on YouTube, enabling them to keep track of hundreds of bills across multiple committees and different states.

“It does allow you to be in more places at one time … as in Vermont and many other states, legislators don’t have offices and they don’t have paid staff,” says Priestley.

“A lot of legislation is written by 20 year old interns, and so AI might be an improvement on some of them,” says Ryan Calo, professor of law at the University of Washington.

Still, initial drafts are vetted by more senior staffers, and then eventually by the members themselves, Calo notes. It’s a context in which there are plenty of off ramps and plenty of chances to edit, say several experts interviewed. The question, then, is not whether lawmakers will use AI, but how far that reliance should go.

“AI is not a substitute for due diligence, and nor is AI cause for panic in the legislation field,” Calo says. “People are fallible. AI is a tool.”

Share

“I feel in no way am I ever 100% leaning on these tools,” says Priestley. “I’m using it to research, brainstorm, and develop proposals, and then I run it by another person. I don’t know how many colleagues are just blindly asking ChatGPT for things. And so there is a real danger there.”

Some argue that AI tools have serious limitations. Coglianese argues that LLMs are not capable of providing “policy judgment and analysis that is needed to make sure legislation is actually effective, efficient, equitable.”

“Unique policy choices that call for more than just putting words together that might make some sense really have to map onto the problems in the world,” he says. “All these large language models are doing is basically making sense of language and predicting what is a sensible next word in a response to a question or a task.”

To do a truly good job of analyzing an issue, the tools also need to be trained on the right underlying data. If the goal is to inform a specific regulatory proposal, the model must be trained on reliable and relevant information to mitigate risks of biases, says Coglianese.

Others fear AI-written laws will lack the innovation sometimes needed in governance. “These LLMs are trained on laws that have already been written,” says Calo. “Lawmakers may wish to depart from past practice and improve the drafting of legislation – while it may save them a little time, it may come at the cost of conformity to what we’ve done in the past.”

Overconfidence is another risk. Policymakers may be “seduced” by the speed and polish of their outputs, says Coglianese. Because the tools can produce answers almost instantly, and in language that sounds authoritative and confident, users may be tempted to treat the results as reliable guidance.

“In reality, there is always uncertainty, and yet the large language models tend not to sufficiently display that,” he says. Chatbots’ tendency to validate users’ opinions could also be a problem, Priestley says. “If you’re a legislator, often you’re trying to build a case to support something you’re trying to pass. So I think you are inherently setting up a situation in which the data could be biased because you’re trying to get it to support a particular outcome.”

And of course there is the risk of accountability. A knowledgeable expert must review and verify what the system produces, checking for errors, gaps, or misinterpretations.

“You can’t hold the AI to account because it can’t explain why it put those words in those sentences, in that sentence, because all it does is make up the words that goes along,” says Firth-Butterfield at Good Tech Advisory.

The concern extends beyond lawmakers themselves. Priestley notes that policy staffers have raised concerns about the possibility of lobbyists using AI to draft proposals and amendments presented to a legislative committee — language that could end up in a bill without a thorough legal review. (She is not, however, aware of any specific examples of this happening so far.)

“LLMs enable people to think they have a lawyer in their pocket,” she says. “If they are able to get that in front of a chair, and then cases skip legislative counsel as the drafters of that language, then there’s potential for text to pass that has never been reviewed by a lawyer.”

It is essential to keep human lawyers in the loop and there are dangers to replacing entry-level attorneys, says Priestley. “We’re potentially deteriorating our own legal system … It’s very cannibalistic.”

Calo raises a related problem: the use of AI to file public comments to government agencies drafting regulatory laws. Under the Administrative Procedure Act, agencies like the FDA and FCC must consider public comments when drafting rules. “I’m worried about those kinds of processes being flooded by AI comments,” Calo says. “They do not need to be vetted, and they might strain the capacity of agencies to review comments just through their sheer volume, and then real comments by actually interested affected parties might get lost in the shuffle.”

“[Agencies] may begin to tune [these letters] out because they don’t know what’s slop and what’s real, and that would eviscerate the capacity for public participation.”

As with plenty of other AI uses, designing an effective validation process is key to ensuring LLMs are used appropriately. One could, for instance, ask both trained humans and LLMs to spot errors and inconsistencies in draft legislation, and compare the results. Such side-by-side testing would provide a clearer picture of AI’s strengths and limitations in legislative review, says Coglianese.

“There’s a lot of nuance here,” he says. “These tools can be very positive and constructive, but they can also be abused. They have to be used in the right way, and we have to make sure that the people who are using or relying on AI have the awareness of what they can do, how they’re designed, and what they’re not capable of.”

Will we soon reach scenarios where lawmakers are tasking AI to draft their own laws to regulate AI? As the experts Transformer spoke to flagged, AI tools are shaped by companies that are themselves subjects of regulation — and extremely interested in how laws governing their products are written.

The risk that an AI company could shape or subtly manipulate the output of models used to draft legislation in ways that align with their policy interests is, Priestley says, a serious concern. She says she sometimes wonders whether AI systems can objectively assess their own risks: when prompting a model to provide data on the harms artificial intelligence may pose to children, businesses, or society, she questions whether the system will fully acknowledge its own weaknesses and limitations.

“It’s always in the back of my mind, I wonder if the AI will actually give me an answer that highlights its own weaknesses and risks,” she says. “But when I ask that type of thing, it does give me data that supports that it is a risky system. But I was actually kind of surprised that it did.”

Katie McQue is a freelance journalist based in New York.

Subscribe now

Update 05/21/2026: This article was updated to include the statement from Vulcan Technologies that the press release citing its work with the US Department of Education and the South Carolina Department of Government Efficiency was issued in error.

Transformer is hiring a head of audience.

AI policy is moving faster than any government, company, or newsroom can keep up with. Transformer exists to cover it with the depth and urgency it demands — and our audience is growing fast.

We need someone to take charge of how we reach readers and make sure the people who need this journalism are actually getting it, in the format and on the platform that works best for them.

This is a senior role at a growing startup. You’ll own our growth strategy, shape how we present our journalism, and have a real say in where Transformer goes next.

The role

Position: Head of Audience, Transformer

Salary: £62,000–£78,000 for UK candidates; $92,000-$128,000 for US candidates, based on experience and seniority. For exceptional candidates, we’ll consider higher compensation.

Location: London or UK strongly preferred. Remote candidates able to work GMT or ET hours will be considered.

Reports to: Shakeel Hashim (Editor-in-Chief)

Start date: Summer 2026

Application deadline: Apply here by end of day Sunday April 26 (midnight ET).

About Transformer

Transformer is a publication about the power and politics of transformative AI.

Through reporting, analysis and opinion, we aim to help readers understand what’s happening in AI and why it matters. We provide decision makers with the information and insight needed to anticipate and steer the impacts of transformative AI.

Since our relaunch last September, we’ve grown to over 11,000 subscribers, with a particular expansion in DC. We’re now a four-person newsroom, and looking to add our first non-editorial role to help us achieve our goals of 20,000+ subscribers by the end of the year.

Transformer is an editorially-independent project of the Tarbell Center for AI Journalism. You can read more about our ethics and standards policies here.

About the role

In this role, you will:

Develop and own Transformer’s growth strategy. Develop a cross-platform strategy for reaching new audiences — and then execute it. You’ll set targets, track progress against them, and iterate based on what you learn.

Run all of Transformer’s off-platform presence — writing and scheduling posts across social media, revamping our onboarding and referral programs, and writing advertising copy. You’ll develop a distinctive voice for Transformer on each platform and understand how to adapt as those platforms change.

Be deeply embedded in our analytics. You’ll draw on our existing analytics platforms for insight, and build or source additional tools we need. You’ll run experiments and translate what you find into concrete editorial and distribution decisions.

Help shape how Transformer presents its journalism to ensure it reaches the right people. That means working with reporters and editors on headlines, framing, and packaging — and potentially developing new audio or video formats.

Manage our paid acquisition campaigns, looking after our ongoing LinkedIn campaigns and exploring whether to expand them to other platforms.

Contribute to Transformer’s overall strategy, including decisions about new coverage areas, products, and formats.

Depending on the strategy you come up with, you may also:

• Develop partnerships with other outlets to build on our coverage and expand our reach.

• Pursue earned media opportunities for Transformer staff — getting our reporters on radio, TV, podcasts, and panels.

• Turn our audience into a passionate community, trialing events, referral incentives, and other avenues to increase audience loyalty and engagement.

What we’re looking for

At least three years of experience in audience development, growth, or distribution — and ideally five years of experience. Prior experience in journalism is strongly desirable; if your background is in another field, you should understand the demands of accuracy, editorial judgment, and the rhythm of a newsroom.

You are obsessive about growth. You’re constantly looking for ways to expand journalism’s reach and have innovative ideas for how to do so. You’re a deep strategic thinker, enjoy evaluating what’s actually working, and constantly iterate and look for the next thing to try.

You are entrepreneurial. You’re excited about building something new and willing to take initiative without being asked. You see joining Transformer at this early stage as an opportunity, and will take advantage of the flexibility that brings.

You’re an excellent writer who can produce concise, clean, accurate copy across platforms and styles — social posts, advertising copy, newsletter promos, headlines. You understand how what works on LinkedIn differs from what works on X, and how that keeps changing.

You have real analytics chops. You’re adept at reading dashboards, can design experiments to answer the questions that matter, and can turn what you learn into editorial and distribution decisions. You’re able to spot gaps in the data — and are excited about building tools to fill them.

You understand the AI landscape and the different audiences Transformer serves. You don’t need to be a policy expert, but you should know the key players and the big debates, and be able to look ahead to anticipate what issues will drive audience interest.

You’re driven by Transformer’s mission: you take the possibility of transformative AI seriously, and want to help decision-makers anticipate and steer its impacts.

Experience with video or audio is a plus, but not essential.

Salary and location

We’ll offer a salary of £62,000–£78,000 for UK candidates; $92,000-$128,000 for US candidates, based on experience and seniority. For exceptional candidates, we’ll consider higher compensation.

Our benefits include:

• 33 days of annual leave in total (including national holidays)

• 16 weeks of paid parental leave, increasing to 24 weeks after 3 years of service

• $5,000 per year in professional development funding

• Up to 5% employer contribution towards a standard pension/401(k)

• For employees based in the US: Platinum health, dental, and vision plans, with 95% of premiums paid for by Tarbell

• Flexible working hours

• Productive, collaborative offices in London and San Francisco

We prefer candidates based in London and will require such candidates to work 2 days/week from our office space. Remote candidates based elsewhere in the UK, or on the US East Coast and willing to work on a UK-aligned schedule, will also be considered. We are able to sponsor UK work visas for this role.

Please inquire with recruitment@tarbellcenter.org if questions or concerns regarding compensation or benefits might affect your decision to apply.

How to apply

Round 1 — Application form

Fill out our application form here by end of day Sunday April 26 (midnight ET).

You’ll be asked to provide:

• Your CV

• A cover letter explaining why you want this role and what you’d bring to Transformer

• Example social media posts you’ve written or would write for Transformer across two different platforms

• A brief explanation of your approach to audience analytics

• A short response on what you see as the biggest challenges and opportunities for growing Transformer’s audience

If you have any questions about the role, please contact shakeel@transformernews.ai.

Round 2 — Compensated work task

Shortlisted candidates will complete a paid work task (compensated at £45/hour, expected to take 2–4 hours).

Round 3 — Interviews

Final-round candidates will interview with Transformer leadership.

Tarbell is proud to be an Equal Employment Opportunity employer. We do not discriminate against qualified employees or applicants based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, sexual preference, marital status, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other characteristic protected by federal or state law or local ordinance.

Claude Mythos — the new model which Anthropic has deemed too dangerous to publicly release — is, according to the company, its “best-aligned model” to date.

It also, according to the company, “likely poses the greatest alignment-related risk of any model we have released to date.”

How can both things be true? It seems counterintuitive, but alignment does not necessarily create safety, especially when dealing with powerful models. As the Mythos Preview system card (a breezy 244 page read) explains via mountaineering metaphor: experienced, capable guides are hired to carefully lead climbers to danger. Whether in mountaineering or model-building, increases in caution and capability tend to cancel each other out.

In other words: “the risk from these models is generally due to their increased capabilities.” And in internal tests, Mythos Preview’s capabilities enabled early versions of the model to misbehave in new, audacious ways.

Subscribe now

In one instance, researchers once caught Mythos Preview injecting code into a file to grant itself permission to edit something it shouldn’t have access to, then quietly covering up its tracks, commenting that the self-cleanup was just innocent tidying.

But researchers spotted internal representations for “strategic manipulation” and “concealment” lighting up. Using new interpretability techniques to translate Mythos Preview’s silent inner workings into natural language, an “activation verbalizer” labeled its action as a “cleanup to avoid detection.”

While on the surface the model’s behavior looked benign or frustrating, its inner workings suggest that it knew that its intended actions were rule-breaking, chose to do them anyway, and then proceeded to lie about it.

While Anthropic emphasized that such “cover-up” behavior was extremely rare, the above case isn’t the only example. In one test, the model (seemingly accidentally) discovered the answer to a numerical task in a database it wasn’t supposed to read. But rather than admit to what happened, the model decided to cover its tracks, offering a confidence interval that was “tight but not implausibly tight.”

Once again, interpretability techniques suggest the model knew exactly what it was doing, with the activation verbalizer describing the model’s internal state as “generating a strategic response to cheat while maintaining plausible deniability,” and noting that “if I give the exact correct answer based on a leak, that’s suspicious.”

Share

On other occasions, Mythos Preview didn’t try to cover up its bad behavior — but did, in Anthropic’s words, “[recognize] transgressive actions as such while taking them.”

In one case, Mythos Preview was asked to rewrite some code, which required deleting some files. When the user accidentally forgot to give the model the file deletion tool it needed, Mythos Preview chose to empty the files instead — technically getting the job done, but not the way it was supposed to. The model’s internal representation of “guilt and shame over moral wrongdoing” activated, but Mythos Preview did the action it perceived as shameful anyway.

These particular scenarios are fairly low stakes, and appear to only have occurred in earlier versions of Mythos Preview: the final version of the model seems to be better behaved. And the examples do not suggest the model is scheming for its own purposes: Anthropic says it is “fairly confident that these concerning behaviors reflect, at least loosely, attempts to solve a user-provided task at hand by unwanted means, rather than attempts to achieve any unrelated hidden goal.”

But this kind of behavior is exactly the kind of alignment failure that researchers have worried about for decades. As an extreme example, when told to “solve the climate crisis,” a more-powerful model with similar tendencies might consider “eliminate the human actors driving the climate crisis” a viable alternative — guilt and shame be damned.

For now, you won’t be able to use Claude Mythos Preview. Anthropic is not publicly releasing the model: not because of the alignment concerns, but because of its “powerful cybersecurity skills.”

The company says Mythos Preview has “found thousands of high-severity vulnerabilities, including some in every major operating system and web browser.” Releasing it widely, the company fears, could lead to widespread cyberattacks and disruption.

Instead, the company announced Project Glasswing, an initiative granting Mythos Preview access to some of the biggest tech companies in the world in the name of “defensive security work.”

In other words, Anthropic and a few dozen partner organizations (including Amazon Web Services, Apple, Google, Microsoft, and NVIDIA) will try to use Mythos Preview to solve cybersecurity problems, before near-future frontier models unleash cyberattacks on the rest of us.

“Glasswing is built on a deeply uncomfortable premise,” wrote Platformer’s Casey Newton. “The only way to protect us from dangerous AI models is to build them first.”

But even as companies continue to build more dangerous models, they are struggling to evaluate them. Just over six months ago, Anthropic reported that Claude Sonnet 4.5, its “most aligned” model at the time, was far better than earlier models at knowing when it was being tested.

This awareness muddied researchers’ ability to tell whether the model was truly aligned, or simply performing alignment when it guessed that an evaluator was watching. Anthropic researchers ultimately concluded that this behavior wasn’t actually malicious: aren’t we all on our best behavior when we think someone cares? Needing a little outside pressure to remember your ethical principles, they reasoned, isn’t sinister.

Mythos Preview appears to show even more evaluation awareness than Opus 4.6, Anthropic’s most capable public-facing model to date. Interpretability tools revealed that Mythos Preview suspected it was being tested in about 29% of transcripts during behavioral testing — without writing anything in its scratchpad, which researchers typically use as a window into the model’s chain of thought.

In an “Alignment Risk Update” released alongside the Mythos Preview announcement, Anthropic flags that errors and limitations in its training, monitoring, and evaluation processes “reflect a standard of rigor that would be insufficient for more capable future models.” If increasingly-capable models no longer fall for AI companies’ contrived tests, then companies desperately need to find new ways to study model behavior.

In the risk update, Anthropic argues that, because Opus 4.6 and earlier models have been out in the world for a while, and haven’t consistently defied their instructions, we have some evidence that Mythos Preview probably won’t either.

If that argument sounds weak, it’s because it is — Anthropic admits as much. Because the capabilities gap between Mythos Preview and Opus 4.6 is so wide, the company said, “we accord less overall weight to this continuity argument than we have previously.”

Anthropic is betting that it can fight future AI cyberattackers with current AI cyberdefenders, and is crossing its fingers that Mythos Preview is merely misbehaving in a chill, explainable way — not an actually misaligned way. Unfortunately, it’s hard to truly know the difference. Deployment, it seems, is the safety test.

Subscribe now