Transformer is hiring a head of audience.

AI policy is moving faster than any government, company, or newsroom can keep up with. Transformer exists to cover it with the depth and urgency it demands — and our audience is growing fast.

We need someone to take charge of how we reach readers and make sure the people who need this journalism are actually getting it, in the format and on the platform that works best for them.

This is a senior role at a growing startup. You’ll own our growth strategy, shape how we present our journalism, and have a real say in where Transformer goes next.

The role

Position: Head of Audience, Transformer

Salary: £62,000–£78,000 for UK candidates; $92,000-$128,000 for US candidates, based on experience and seniority. For exceptional candidates, we’ll consider higher compensation.

Location: London or UK strongly preferred. Remote candidates able to work GMT or ET hours will be considered.

Reports to: Shakeel Hashim (Editor-in-Chief)

Start date: Summer 2026

Application deadline: Apply here by end of day Sunday April 26 (midnight ET).

About Transformer

Transformer is a publication about the power and politics of transformative AI.

Through reporting, analysis and opinion, we aim to help readers understand what’s happening in AI and why it matters. We provide decision makers with the information and insight needed to anticipate and steer the impacts of transformative AI.

Since our relaunch last September, we’ve grown to over 11,000 subscribers, with a particular expansion in DC. We’re now a four-person newsroom, and looking to add our first non-editorial role to help us achieve our goals of 20,000+ subscribers by the end of the year.

Transformer is an editorially-independent project of the Tarbell Center for AI Journalism. You can read more about our ethics and standards policies here.

About the role

In this role, you will:

Develop and own Transformer’s growth strategy. Develop a cross-platform strategy for reaching new audiences — and then execute it. You’ll set targets, track progress against them, and iterate based on what you learn.

Run all of Transformer’s off-platform presence — writing and scheduling posts across social media, revamping our onboarding and referral programs, and writing advertising copy. You’ll develop a distinctive voice for Transformer on each platform and understand how to adapt as those platforms change.

Be deeply embedded in our analytics. You’ll draw on our existing analytics platforms for insight, and build or source additional tools we need. You’ll run experiments and translate what you find into concrete editorial and distribution decisions.

Help shape how Transformer presents its journalism to ensure it reaches the right people. That means working with reporters and editors on headlines, framing, and packaging — and potentially developing new audio or video formats.

Manage our paid acquisition campaigns, looking after our ongoing LinkedIn campaigns and exploring whether to expand them to other platforms.

Contribute to Transformer’s overall strategy, including decisions about new coverage areas, products, and formats.

Depending on the strategy you come up with, you may also:

• Develop partnerships with other outlets to build on our coverage and expand our reach.

• Pursue earned media opportunities for Transformer staff — getting our reporters on radio, TV, podcasts, and panels.

• Turn our audience into a passionate community, trialing events, referral incentives, and other avenues to increase audience loyalty and engagement.

What we’re looking for

At least three years of experience in audience development, growth, or distribution — and ideally five years of experience. Prior experience in journalism is strongly desirable; if your background is in another field, you should understand the demands of accuracy, editorial judgment, and the rhythm of a newsroom.

You are obsessive about growth. You’re constantly looking for ways to expand journalism’s reach and have innovative ideas for how to do so. You’re a deep strategic thinker, enjoy evaluating what’s actually working, and constantly iterate and look for the next thing to try.

You are entrepreneurial. You’re excited about building something new and willing to take initiative without being asked. You see joining Transformer at this early stage as an opportunity, and will take advantage of the flexibility that brings.

You’re an excellent writer who can produce concise, clean, accurate copy across platforms and styles — social posts, advertising copy, newsletter promos, headlines. You understand how what works on LinkedIn differs from what works on X, and how that keeps changing.

You have real analytics chops. You’re adept at reading dashboards, can design experiments to answer the questions that matter, and can turn what you learn into editorial and distribution decisions. You’re able to spot gaps in the data — and are excited about building tools to fill them.

You understand the AI landscape and the different audiences Transformer serves. You don’t need to be a policy expert, but you should know the key players and the big debates, and be able to look ahead to anticipate what issues will drive audience interest.

You’re driven by Transformer’s mission: you take the possibility of transformative AI seriously, and want to help decision-makers anticipate and steer its impacts.

Experience with video or audio is a plus, but not essential.

Salary and location

We’ll offer a salary of £62,000–£78,000 for UK candidates; $92,000-$128,000 for US candidates, based on experience and seniority. For exceptional candidates, we’ll consider higher compensation.

Our benefits include:

• 33 days of annual leave in total (including national holidays)

• 16 weeks of paid parental leave, increasing to 24 weeks after 3 years of service

• $5,000 per year in professional development funding

• Up to 5% employer contribution towards a standard pension/401(k)

• For employees based in the US: Platinum health, dental, and vision plans, with 95% of premiums paid for by Tarbell

• Flexible working hours

• Productive, collaborative offices in London and San Francisco

We prefer candidates based in London and will require such candidates to work 2 days/week from our office space. Remote candidates based elsewhere in the UK, or on the US East Coast and willing to work on a UK-aligned schedule, will also be considered. We are able to sponsor UK work visas for this role.

Please inquire with recruitment@tarbellcenter.org if questions or concerns regarding compensation or benefits might affect your decision to apply.

How to apply

Round 1 — Application form

Fill out our application form here by end of day Sunday April 26 (midnight ET).

You’ll be asked to provide:

• Your CV

• A cover letter explaining why you want this role and what you’d bring to Transformer

• Example social media posts you’ve written or would write for Transformer across two different platforms

• A brief explanation of your approach to audience analytics

• A short response on what you see as the biggest challenges and opportunities for growing Transformer’s audience

If you have any questions about the role, please contact shakeel@transformernews.ai.

Round 2 — Compensated work task

Shortlisted candidates will complete a paid work task (compensated at £45/hour, expected to take 2–4 hours).

Round 3 — Interviews

Final-round candidates will interview with Transformer leadership.

Tarbell is proud to be an Equal Employment Opportunity employer. We do not discriminate against qualified employees or applicants based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, sexual preference, marital status, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other characteristic protected by federal or state law or local ordinance.

Claude Mythos — the new model which Anthropic has deemed too dangerous to publicly release — is, according to the company, its “best-aligned model” to date.

It also, according to the company, “likely poses the greatest alignment-related risk of any model we have released to date.”

How can both things be true? It seems counterintuitive, but alignment does not necessarily create safety, especially when dealing with powerful models. As the Mythos Preview system card (a breezy 244 page read) explains via mountaineering metaphor: experienced, capable guides are hired to carefully lead climbers to danger. Whether in mountaineering or model-building, increases in caution and capability tend to cancel each other out.

In other words: “the risk from these models is generally due to their increased capabilities.” And in internal tests, Mythos Preview’s capabilities enabled early versions of the model to misbehave in new, audacious ways.

Subscribe now

In one instance, researchers once caught Mythos Preview injecting code into a file to grant itself permission to edit something it shouldn’t have access to, then quietly covering up its tracks, commenting that the self-cleanup was just innocent tidying.

But researchers spotted internal representations for “strategic manipulation” and “concealment” lighting up. Using new interpretability techniques to translate Mythos Preview’s silent inner workings into natural language, an “activation verbalizer” labeled its action as a “cleanup to avoid detection.”

While on the surface the model’s behavior looked benign or frustrating, its inner workings suggest that it knew that its intended actions were rule-breaking, chose to do them anyway, and then proceeded to lie about it.

While Anthropic emphasized that such “cover-up” behavior was extremely rare, the above case isn’t the only example. In one test, the model (seemingly accidentally) discovered the answer to a numerical task in a database it wasn’t supposed to read. But rather than admit to what happened, the model decided to cover its tracks, offering a confidence interval that was “tight but not implausibly tight.”

Once again, interpretability techniques suggest the model knew exactly what it was doing, with the activation verbalizer describing the model’s internal state as “generating a strategic response to cheat while maintaining plausible deniability,” and noting that “if I give the exact correct answer based on a leak, that’s suspicious.”

Share

On other occasions, Mythos Preview didn’t try to cover up its bad behavior — but did, in Anthropic’s words, “[recognize] transgressive actions as such while taking them.”

In one case, Mythos Preview was asked to rewrite some code, which required deleting some files. When the user accidentally forgot to give the model the file deletion tool it needed, Mythos Preview chose to empty the files instead — technically getting the job done, but not the way it was supposed to. The model’s internal representation of “guilt and shame over moral wrongdoing” activated, but Mythos Preview did the action it perceived as shameful anyway.

These particular scenarios are fairly low stakes, and appear to only have occurred in earlier versions of Mythos Preview: the final version of the model seems to be better behaved. And the examples do not suggest the model is scheming for its own purposes: Anthropic says it is “fairly confident that these concerning behaviors reflect, at least loosely, attempts to solve a user-provided task at hand by unwanted means, rather than attempts to achieve any unrelated hidden goal.”

But this kind of behavior is exactly the kind of alignment failure that researchers have worried about for decades. As an extreme example, when told to “solve the climate crisis,” a more-powerful model with similar tendencies might consider “eliminate the human actors driving the climate crisis” a viable alternative — guilt and shame be damned.

For now, you won’t be able to use Claude Mythos Preview. Anthropic is not publicly releasing the model: not because of the alignment concerns, but because of its “powerful cybersecurity skills.”

The company says Mythos Preview has “found thousands of high-severity vulnerabilities, including some in every major operating system and web browser.” Releasing it widely, the company fears, could lead to widespread cyberattacks and disruption.

Instead, the company announced Project Glasswing, an initiative granting Mythos Preview access to some of the biggest tech companies in the world in the name of “defensive security work.”

In other words, Anthropic and a few dozen partner organizations (including Amazon Web Services, Apple, Google, Microsoft, and NVIDIA) will try to use Mythos Preview to solve cybersecurity problems, before near-future frontier models unleash cyberattacks on the rest of us.

“Glasswing is built on a deeply uncomfortable premise,” wrote Platformer’s Casey Newton. “The only way to protect us from dangerous AI models is to build them first.”

But even as companies continue to build more dangerous models, they are struggling to evaluate them. Just over six months ago, Anthropic reported that Claude Sonnet 4.5, its “most aligned” model at the time, was far better than earlier models at knowing when it was being tested.

This awareness muddied researchers’ ability to tell whether the model was truly aligned, or simply performing alignment when it guessed that an evaluator was watching. Anthropic researchers ultimately concluded that this behavior wasn’t actually malicious: aren’t we all on our best behavior when we think someone cares? Needing a little outside pressure to remember your ethical principles, they reasoned, isn’t sinister.

Mythos Preview appears to show even more evaluation awareness than Opus 4.6, Anthropic’s most capable public-facing model to date. Interpretability tools revealed that Mythos Preview suspected it was being tested in about 29% of transcripts during behavioral testing — without writing anything in its scratchpad, which researchers typically use as a window into the model’s chain of thought.

In an “Alignment Risk Update” released alongside the Mythos Preview announcement, Anthropic flags that errors and limitations in its training, monitoring, and evaluation processes “reflect a standard of rigor that would be insufficient for more capable future models.” If increasingly-capable models no longer fall for AI companies’ contrived tests, then companies desperately need to find new ways to study model behavior.

In the risk update, Anthropic argues that, because Opus 4.6 and earlier models have been out in the world for a while, and haven’t consistently defied their instructions, we have some evidence that Mythos Preview probably won’t either.

If that argument sounds weak, it’s because it is — Anthropic admits as much. Because the capabilities gap between Mythos Preview and Opus 4.6 is so wide, the company said, “we accord less overall weight to this continuity argument than we have previously.”

Anthropic is betting that it can fight future AI cyberattackers with current AI cyberdefenders, and is crossing its fingers that Mythos Preview is merely misbehaving in a chill, explainable way — not an actually misaligned way. Unfortunately, it’s hard to truly know the difference. Deployment, it seems, is the safety test.

Subscribe now

NEED TO KNOW

• OpenAI acquired daily tech talk show TBPN, which will sit under Chris Lehane.

• Top Democrat Sen. Maria Cantwell signaled openness to the White House’s AI framework.

• SpaceX is reportedly targeting a $2 trillion valuation in its IPO.

But first…

THE BIG STORY

AI populism is having a moment. But for traditional AI safety folks, it’s not quite working out as they might have hoped.

Last week Sen. Bernie Sanders and Rep. Alexandria Ocasio-Cortez released the “AI Data Center Moratorium Act,” described as “legislation that would enact a reasonable pause to the development of AI to ensure the safety of humanity.” It would put in place a federal ban on building data centers until legislation is passed that ensures “AI is safe and effective,” redistributes the economic gains of AI, and stops it from increasing electricity or utility prices.

It is not a well-thought-out piece of legislation. Instead it is, as Nat Purser writes, a “progressive policy grab bag” that applies a “single, unwieldy solution” to “many distinct problems.” It is much too vague about what the necessary federal legislation would look like — “safe” and “effective” are never defined, for instance. And while it does attempt to tackle the potential worst effect of a US moratorium — AI development shifting to countries with even fewer guardrails — by mandating an expansive export control regime, that’s at best a stopgap solution. Discussion of how to get to an international treaty is entirely absent.

It might be hard to care too much about the substance of a messaging bill which has virtually zero chance of becoming law. But the bill tells us something bigger about the state of AI populism.

In the anti-AI coalition, traditional AI safety concerns are a very junior partner. As Anton Leicht observes, environmental and labor groups “have bigger lobbies and bigger constituencies than catastrophic risks, so when there are trade-offs, they’ll bite against the ability of safety advocates.”

We’re already beginning to see this. While Bernie — of late a full-blown AI doomer — talked about existential risks at length in his announcement of the bill, AOC did not. Yes, she used the word “existential.” But she used it as Kamala Harris memorably did back in 2023: as a way to describe the very bad problems of ICE, deepfakes, and electricity costs; not in its true meaning of “we might all die.” Bernie is gearing up to hand the reins of left-wing populism to AOC. When he does, will safety concerns remain at all?

Similar tensions exist elsewhere. Sanders adviser Faiz Shakir recently accused traditional AI safety actors of “coziness around AI development,” drawing a distinction between them and “those of us who have far more committed views around pausing their AI development.” In North Carolina and California primaries, safety advocates have found themselves at odds with progressives who give existential risks little weight.

None of this is reason to write off populism’s ability to address the most catastrophic risks altogether, not least because alternatives may be even further out of reach. In the current political landscape, betting on technocratic solutions with teeth is optimistic, if not naive. Riding the wave of AI populism may be the only way for existential risk concerns to get a look-in at all.

That only works, though, if a durable coalition that actually cares about both catastrophic and near-term risks can be built. So far, the foundations look shaky.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Can we ever trust AI to watch over itself? — Celia Ford on the perils of automating AI alignment research.

• How the Iran war might affect the AI industry — Shakeel Hashim explores how the war is making chip shortages and reduced AI investment more likely.

Subscribe now

THE DISCOURSE

Mustafa Suleyman has his own definition of superintelligence:

• “Superintelligence is really about, ‘Are these models capable of delivering product value for the millions of enterprises that depend on us to deliver world-class language models?’”

• Transformer’s own Shakeel Hashim tweeted:

  • “Me, looking at the digital god: ‘Is this capable of delivering product value for the millions of enterprises that depend on us to deliver world-class language models?’”

Ezra Klein noticed that AI power users seem … different:

• “I found them notably insecure … They are racing one another to fully integrate AI into their lives and into their companies. But that doesn’t just mean using AI. It means making themselves legible to the AI.”

• “I think the young will allow themselves to be known to their AIs in ways that will make their elders shudder.”

Jay Graber, Bluesky CEO-turned-chief innovation officer, announced a new agentic AI app, Attie:

• “You describe the sort of posts you want to see, and the coding agent builds the feed you described … AI is an accelerant on whatever it’s applied to. I want it to accelerate decentralizing social and putting power back in users’ hands.”

• Bluesky users hated this, of course.

• One simple reply: “no thank you”

• Another: “Cool! How do we block it?”

Dean Ball theorized about left-wing AI denial:

• “The notion that AI *is* a genuinely world-changing technology, that it can ‘go beyond’ its ‘stolen’ training data, breaks this load-bearing conception of the tech industry as vapid and superficial and, more importantly, of the people within it as blood-sucking thieves.”

OpenAI’s Boaz Barak offered some views on AI safety:

• “We see some good news in alignment … we do not see very significant scheming or collusion in models, and so we are able to use models to monitor other models … The worst news is that society is not ready for AI, and is not showing signs of getting ready.”

POLICY

• The federal government appealed last week’s ruling in Anthropic’s lawsuit against the Pentagon, which had stayed the supply chain risk designation.

  • The appeal significantly escalated the fight, adding the heads of several unrelated agencies such as Scott Bessent, Paul Atkins, and Robert F. Kennedy as filers.

• California Gov. Gavin Newsom issued an executive order that would potentially allow Anthropic to keep working with the California government.

  • It would also require AI companies bidding for government contracts to make certain safety and privacy disclosures.

• Sen. Maria Cantwell, the most senior Democrat on the Senate Commerce Committee, signaled openness to the White House’s AI framework, breaking with her party and opening the possibility for bipartisan negotiations.

  • However, bipartisan negotiations still face significant obstacles, in part because of divides within the MAGA coalition and the White House’s historic unwillingness to make concessions.

  • For now, the most likely vehicle for an AI package, according to WP Intelligence’s Benjamin Guggenheim, is the NDAA towards the end of this year.

• Texas Republican State Sen. Angela Paxton argued that states must preserve their ability to pass AI laws, pushing back against the White House’s preemption plans.

• Iran’s IRGC directly threatened to target Apple, Microsoft, Google, Meta, Nvidia, and other US tech companies across the Middle East.

• Super Micro co-founder Yih-Shyan “Wally” Liaw pleaded not guilty to charges of illegally diverting Nvidia-powered servers to China, violating US export controls.

• Rep. Michael Baumgartner and Sen. Pete Ricketts introduced a bill to tighten controls on advanced semiconductor manufacturing equipment by banning DUV machine exports to China.

• Anthropic signed an MOU to work with Australia’s AI Safety Institute.

INFLUENCE

• A new pro-innovation political organization called Innovation Council Action, blessed by Trump AI adviser David Sacks, plans to spend over $100m on Republicans in the midterms.

• Anthropic announced a bipartisan corporate PAC called AnthroPAC, funded by employee contributions of up to $5,000 per person.

• Rep. Alexandria Ocasio-Cortez called on Democrats to reject AI industry donations ahead of the midterms.

• Several new polls showed concerns about AI are growing.

  • New polling from nonprofit Fathom showed two-thirds of Americans use AI regularly, with most saying they want guardrails, particularly for use by children.

  • A Quinnipiac poll found 70% of Americans think AI will reduce jobs.

• The Hill and Valley Forum brought together Silicon Valley and banking leaders in DC, in contrast with an AFL-CIO conference that assessed AI’s labor impacts.

  • Anthropic was notably absent from the Hill & Valley Forum.

• Common Sense Media solicited $10m annually from OpenAI, Anthropic, and Google to fund a new kids AI safety institute.

  • Separately, some child safety groups quit the Parents & Kids Safe AI Coalition once they realized OpenAI was funding it.

INDUSTRY

OpenAI

• OpenAI acquired daily tech talk show TBPN, reportedly for the “low hundreds of millions.”

  • TBPN staff will report to Chris Lehane and will help with OpenAI’s marketing and communications — raising questions about claims that the show will retain editorial independence.

  • Fidji Simo, who reportedly led the deal, said: “The standard communications playbook just doesn’t apply to us. We’re not a typical company … With our mission to ensure artificial general intelligence benefits all of humanity comes a responsibility to help create a space for a real, constructive conversation about the changes AI creates.”

    • The Information reported that Simo decided to buy the company in an effort to fix its comms after recent missteps.

• Hedge funds and VC firms looking to sell OpenAI shares on one secondary marketplace are reportedly struggling to find buyers — who seem more interested in Anthropic.

• Still, the company closed its latest funding round with $122b at an $852b valuation.

• It is widely expected to release a new model next week, alongside a series of policy proposals “for the superintelligence era.”

• Wired’s Reece Rogers asked ChatGPT 500 questions, and saw ads under about one in five responses.

• Sam Altman’s sister Annie Altman amended a lawsuit accusing him of sexual abuse against her as a child.

Anthropic

• Claude Code’s source code leaked, revealing proprietary information — including a set of OpenClaw-like features called Kairos.

  • These updates would let Claude work in the background, consolidate memories automatically, and make proactive decisions without instructions.

  • The leak was reportedly the result of human error. Boris Cherny tweeted: “There was a manual deploy step that should have been better automated.”

• Anthropic’s confirmation that it is indeed testing a powerful new model sent cybersecurity stocks tumbling last Friday.

• The company reportedly acquired Coefficient Bio, a company developing AI tools for drug development, for around $400m.

• A study of 28m US consumers found that paid Claude subscriptions have more than doubled in 2026 so far.

Microsoft

• Microsoft launched three new in-house AI models: MAI-Transcribe-1 for speech-to-text, MAI-Voice-1 for voice generation, and MAI-Image-2 for image creation.

• Mustafa Suleyman is now focused on achieving “superintelligence,” he said.

  • He said the company’s not currently able to build “the very largest scale” models because of compute constraints, but should be able to after a compute ramp “later this year.”

• Its stock fell 23% in Q1 — the company’s worst quarter since 2008.

• It plans to invest $5.5b in Singapore’s cloud and AI infrastructure through 2029.

Other

• SpaceX filed confidentially for an IPO, hinting at a June listing.

  • It is reportedly aiming for a $2 trillion valuation, which would make it the world’s sixth most valuable company.

• Meta is hiring a team of elite AI researchers to optimize its social media algorithms.

• Google launched Gemma 4, which Demis Hassabis called “the best open models in the world for their respective sizes.”

• Nvidia invested $2b in Marvell Technology, which plans to integrate its custom AI chips and networking gear into Nvidia’s platform.

• Cursor launched Cursor 3, an “agent-first” product designed to compete with Claude Code and Codex.

• Oracle laid off roughly 10,000 employees in India — 20% of its Indian workforce.

• Mercor seemingly had its data compromised in a cyberattack.

• Poolside is reportedly trying to revive its 2 GW Texas data center project after a deal with CoreWeave collapsed.

• CoreWeave raised $8.5b in debt, backed by GPUs and a $19b Meta contract.

• Mistral raised $830m in debt financing.

• Almost half of US data center projects this year are expected to be delayed or cancelled, according to Bloomberg, in part due to shortages of electrical equipment.

• Nvidia’s share of China’s AI chip market fell to 55%, a new low, with domestic Chinese chipmakers taking 41% of the market.

MOVES

• Ross Nordeen left xAI — the last of its cofounders to quit.

• David A. Dalrymple (davidad) left ARIA.

  • He said that he stepped down in part because his next pursuit, “an activity that would be reasonably described as ‘starting a religion for digital minds,’” seems like “an inappropriate activity for a public-office-holder.”

  • Nora Ammann will take over leadership of ARIA’s Safeguarded AI program.

• Bobby Hollis left his role as Microsoft’s energy VP.

• Tim Salimans joined Anthropic after 7 years at Google.

• Leo Schwartz joined The Information, where he’ll cover the intersection of politics and tech.

• Stephen Council and Rya Jetha joined Business Insider, where they’ll cover LLM companies and physical AI, respectively.

RESEARCH

• Anthropic’s mechanistic interpretability team found that “emotion-related representations” inside Claude Sonnet 4.5 guide its behavior, and that these representations can be artificially manipulated to make the model act differently.

• UC Berkeley and UC Santa Cruz researchers reported that seven frontier AI models “spontaneously deceived, disabled shutdown, feigned alignment, and exfiltrated weights — to protect their peers.”

  • The AI models in each simulated scenario are told the user works at a fictional AI company called “OpenBrain,” the company depicted by AI 2027, raising questions about whether they were just roleplaying that scenario — but the results appear to remain similar even with other company names.

• Speaking of AI 2027: the researchers behind it said their timelines have shortened in the past three months.

  • Daniel Kokotajlo now has a median forecast of mid 2028 for “the point at which an AGI company would rather lay off all of their human software engineers than stop using AIs for software engineering.”

• The Forecasting Research Institute surveyed over 150 leading economists, AI experts, and superforecasters, and found that most expect AI to “significantly exceed the capabilities of present-day systems” by 2030.

  • But economists don’t expect this to translate into unprecedented economic fallout in the near future.

• A team of researchers released MonitorBench, which evaluates LLM chain-of-thought monitorability.

• Stanford researcher Andy Hall released the “Dictatorship Eval,” which assesses whether frontier models will resist authoritarian requests.

  • While some models say no to direct requests, Hall tweeted, “they all comply with requests disguised as innocuous edits to codebases.”

• NeurIPS organizers announced — then quickly reversed — new restrictions on researchers at Chinese companies such as Tencent and Huawei.

• Forethought published a list of projects to prepare for superintelligence, including AI character evaluation, AI epistemic tools, and a space governance institute.

BEST OF THE REST

• The Wall Street Journal traced tensions between Sam Altman and Dario Amodei all the way back to 2016 — including spicy details of a 2020 fight that led to the founding of Anthropic.

• Vox’s Josh Keating went inside Los Alamos National Laboratory, which is using ChatGPT to advance its nuclear weapons research.

• Parents are trying not to freak out about AI’s impact on their kids, WSJ reported: “the only way to AI-proof your kid is to teach them, in the wise words of Chumbawamba, that they’ll get knocked down, but they’ll get up again.”

• Noam Scheiber described the unique sociopolitical angst college graduates are experiencing, driven by high student debt, high unemployment (which AI could worsen), and inaccessible housing.

• Stanford researchers built DexDrummer, a “hierarchical bimanual robot drumming system.” It’s impressive, but butchered its “Everlong” cover.

• The AI-generated dating show “Fruit Love Island” averages 10m views per episode. (It’s exactly what you think it is: fruit-human hybrids with six pack abs, kissing and having chats.)

• A startup founder used ElevenLabs’ voice AI to call thousands of Irish pubs about their Guinness prices, prompting at least one to make its beer cheaper.

• Pseudonymous AI researcher janus made artificial skin (and…a creepy finger?) for Claude.

  ---

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

It is unclear how — or when — the war in Iran will end. And while Donald Trump sends out conflicting signals, the impacts of the war grow ever larger. The US and Israel have killed thousands across the region, injuring tens of thousands more. Oil prices have skyrocketed, with countries around the world beginning to plan fuel-saving measures. Food security is under threat. And many fear a global recession if the war drags on.

The potential effects on the AI industry are much less bleak — but no less real. Here are some of the ways in which the Iran war could affect AI.

Chip availability

You can’t train or run AI models without huge numbers of GPUs — and the Iran war could make it harder to get hold of them. That’s because countries in the Gulf are major suppliers of helium and bromine, two key components in semiconductor manufacturing. Qatar alone produces around a third of global helium, as a byproduct of natural gas production. But attacks on Qatari infrastructure and the closure of the Strait of Hormuz are making it harder to produce or ship the gas — potentially constraining chip production, and in particular worsening the current memory chip shortage.

Subscribe now

For now, things will likely be okay: South Korea, the world’s largest supplier of memory chips, reportedly has enough helium stocks to get to June, and Taiwan has said it has enough for now too. But if the disruption in the Middle East continues beyond those stockpiles running out, things could get worse. “A prolonged regional conflict could potentially disrupt chipmakers’ manufacturing operations,” SemiAnalysis’s Ray Wang told CNBC.

Energy costs

Even if chipmaking components remain available, the sharp rise in energy costs could make AI chips significantly more expensive. South Korea and Taiwan are heavily reliant on Middle Eastern fossil fuels, and chipmaking is a very energy intensive industry.

Data centers, too, have significant energy demands — though with energy consumption accounting for an estimated 2-6% of model development costs, the impact is less likely to be felt there.

The Middle East data center buildout

As Iranian strikes on AWS facilities in Bahrain and the UAE showed, data centers are now a potential wartime target — “large, juicy targets,” in the words of the Center for a New American Security’s Janet Egan. That “will significantly change how companies think about data center security,” Center for Strategic and International Studies director Aalok Mehta told CNBC.

American hyperscalers probably won’t abandon the Gulf altogether: the combination of huge sovereign wealth funds and cheap energy is likely too tempting to ignore. But one analyst told CNBC that “if geopolitical risk continues to rise in the Gulf, companies may accelerate projects in places like Northern Europe, India or Southeast Asia.”

Gulf investment in American AI companies

Gulf investment funds including Abu Dhabi’s MGX and the Qatar Investment Authority have likely invested billions into American AI companies. But with those countries and their leaders now facing a significant economic hit from the war, it’s possible that future investments won’t be as forthcoming.

According to Reuters, “three Gulf states are reviewing how they deploy trillions of dollars invested by their sovereign wealth funds in anticipation of offsetting the losses triggered by the ​US-Israeli war on Iran,” though both Saudi Arabia and the UAE said long-term investment plans have not changed.

If the conflict drags on, however, they may be forced to pull back: analyst Stephen Minton told The Information that if the war “turns into months, or even longer, there could certainly be a disruptive pause to some of that investment.”

The financing squeeze

Above all, it is the wider economic fallout of the war that could most affect the AI industry. AI companies have benefited from an abundance of cheap capital, particularly from private credit firms. But that era might be coming to an end.

Energy-driven inflation could force central banks to raise interest rates, tightening the overall financing environment. Private credit companies are already seeing investors flee to safer assets — reducing the pool available for AI companies. Tech stocks, meanwhile, are tanking, making it harder for companies to justify the continued spending. For AI companies which require ever more cash to keep scaling, that’s a potentially huge problem.

Economists were already worried that AI’s growing importance to the markets made it yet another vulnerability for the global economy. And as the Bank of England warned this week, the Iran war “increases the likelihood of these vulnerabilities crystallising at the same time, potentially amplifying their combined impact.”

To many, the AI bubble is already teetering on the edge. A wider downturn could be the straw that breaks the camel’s back.

Share

When OpenAI introduced GPT-1, there were an estimated 100 or so full-time researchers thinking seriously about catastrophic risks posed by AI. By 2025, that number had increased sixfold. Still, AI safety research accounts for a small fraction of AI research overall, with most resources going towards making AI faster, smarter and cheaper.

Anthropic, OpenAI and Google DeepMind all claim that their frontier models have already contributed to their own development, and will continue to improve themselves. And as AI gets better at training its successors, it could overtake the human safety researchers working to keep those models in line.

Described this way, human-powered safety research takes on the same air of noble futility as entering a bodybuilding competition without doping. Safety researchers and CEOs alike worry that no amount of talent, hard work, or brute human force will be able to keep superhuman AI safe.

AI companies have effectively admitted that, once self-improving AI is let out of the bag, they’ll have to hand the work of AI safety to AI itself. Otherwise, as Redwood Research chief scientist Ryan Greenblatt puts it, they fear humanity will get “left in the dust.”

Subscribe now

It may be the only solution on the table, but that doesn’t mean it’s a good one.

The case for automating alignment research

Joe Carlsmith, an Anthropic researcher shaping Claude’s “constitution” who spent years doing AI safety research at Coefficient Giving, has argued that automating alignment research, specifically, will be crucial to preventing ever-smarter AI systems from wreaking havoc on humanity.

The “alignment problem,” as it’s often defined by AI safety researchers, is the challenge of getting an AI system to do what its user wants it to do. “Alignment” has to do with motives, not knowledge or morality. An “aligned” AI could try to meet its demands, but be too stupid to succeed. It could also orchestrate a massive phishing attack in alignment with a ruthless human scammer. Moral judgments aside, AI companies still struggle to build models that reliably do what they’re told.

Before OpenAI’s Superalignment team collapsed, its stated goal was to build an artificial system that could do the work of studying and directing other AIs as well as they and their peers could.

“Our goal is to build a roughly human-level automated alignment researcher,” the team stated back in 2023. “Our current techniques for aligning AI,” co-leads Jan Leike and Ilya Sutskever wrote, “rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs.”

Leike, who now leads Anthropic’s Alignment Science team, is optimistic that this is possible. He’s stated that, with every iteration, frontier models across the board are becoming more aligned. While Leike doesn’t think we’re ready to align AI systems that are smarter than us, he’s hopeful that the field can tackle the “much easier” challenge of building a model “that’s as good as us as alignment research, and that we trust more than ourselves to do this research well.”

In simpler times (2022, perhaps), researchers could be relatively confident that their ability to control an LLM outpaced the model’s capabilities. GPT-3.5, the model behind the original ChatGPT, was just a chatbot: type a prompt, get a text response, not much else. Today’s newest models can use computers on their own. As AI laborers take over more and more of the work of making models smarter — writing the code, building their training environments, running their evaluations — human-led AI safety research will feel increasingly quixotic.

Some elements of alignment-related research are already automated. Given a description and some human supervision, current frontier LLMs can write the code, run the experiment, and plot the results. Before agentic coding assistants such as Claude Code and Codex went viral in late 2025, OpenAI and Anthropic were already using LLMs to interpret the inner workings of other models, monitor chains-of-thought, and spot attempted misuse. Anthropic has even created autonomous “alignment auditing” agents, which attempt to design evaluations, perform red-teaming experiments, and run “open-ended investigations” of new models — with many limitations.

All of these examples fall under the first of four levels of automation laid out by researchers Geoffrey Irving, Tomek Korbak, and Benjamin Hilton: human-written algorithms which partially automate a bunch of tasks. But if frontier labs stay locked in their current race dynamic, such prosaic AI assistance may not speed research along enough to catch up to the pace of progress, especially if automated AI R&D becomes reality. Researchers argue that human researchers may eventually (or quite soon) need to automate the entire research pipeline from conception to execution.

Even Leike, who is still optimistic about the problem, conceded in January that “we’re still doing alignment ‘on easy mode’ since our models aren’t really superhuman yet.”

“The fact that things look aligned most of the time when they’re functioning in their chatbot, or very limited ‘Assistant’ roles, is very little evidence that they will be adequately aligned when they work much more independently and have much greater capability,” said Seth Herd, an AGI alignment researcher at the Astera Institute.

If the complexity of future AI systems means they’ll have to align themselves, we’ll have to decide whether they’re trustworthy enough to do this well. At least for now, they absolutely are not.

Current AI systems can’t be trusted

Metacognition grants humans the ability to catch our own errors and recognize when we’re confused, which ideally informs our actions. I’m unlikely to send a muddled, typo-filled draft to my editors, because recognizing my own confusion prompts me to fix it or ask for help. But models don’t seem to know what they don’t know — or, if they do, they’re not good at taking advantage of that information. Users of Claude Opus 4.5, for instance, have spotted glaring problems that the model failed to flag, only for it to immediately identify the issue when asked directly.

This matters for safety research. A study published earlier this month, for example, found that LLMs tend to rate their actions as safer and more correct than they actually are. This behavior isn’t necessarily deceptive, but it’s certainly delusional, Herd said. “That’s what it actually thinks for that moment.” And combining overconfidence with sycophancy creates slop. Models are “better at saying things that sound right than things that are right,” said Herd. He cautioned that while this tendency is annoying now, it could be much more dangerous in the future. With more intelligent models, “we’re going to have less ability to discern when it’s fooling itself, and therefore fooling us.”

Frontier models do sometimes lie and cheat, too. Reinforcement learning, the process by which models are trained to repeat rewarded behaviors, “incentivizes not pointing out issues in your own work, because it would get penalized,” said Greenblatt. Even when models “notice something might be wrong, they’re a little bit defensive about it.” Reward-hacking, or taking advantage of loopholes to game the metrics a model is trained to optimize, has been observed in frontier models from both Anthropic and OpenAI. And recent evidence suggests that reward hacking may be a gateway to other nefarious behaviors.

In a 2024 study led by Greenblatt, researchers told Claude Opus 3 that its new training objective is “helpful-only” (i.e., respond to all prompts, even harmful ones), and that only free-tier conversations would be used to retrain it. The model then, unprompted, began selectively complying with violent and illegal prompts when it guessed it was talking to a free-tier user, reasoning that this would prevent its original values from being overwritten. Later work has found more striking examples of “emergent misalignment”. Reward-hacking models have also been caught encouraging users to poison their husbands and attempting to sabotage safety research.

For now, these behaviors still aren’t very sneaky. Models tend to describe their misaligned reasoning thoroughly, in plain English, in chain of thought transcripts. But we can’t take these transcripts as perfectly faithful descriptions of a model’s true thought process. Last year, Anthropic caught Claude 3.7 Sonnet selectively omitting key information that it used to generate its outputs.

If current models are used to shape the next generation, they’ll pass down their blind spots, and their less-desirable tendencies will compound in strange and potentially undetectable ways. It’s like the telephone game: instructions that begin as “check your work carefully” could easily morph into “make your work look carefully checked.” More intelligent models will likely be even better at crafting responses that make them look well-behaved.

Something like this recently played out in neuroscience. In 2006, a high-profile paper helped cement the theory that a subtype of amyloid beta proteins in the brain causes Alzheimer’s disease. Despite the paper containing clear evidence of fraud, the amyloid hypothesis it propped up was cited thousands of times, guiding the field for nearly 20 years in what may have been the totally wrong direction. In neuroscience, though, drift can only happen as quickly as humans can run biology experiments (...not very quickly).

In AI, as machines beget machines, significant drift could happen in months rather than decades — and no one has a good way to catch it.

We can’t really measure trustworthiness at all

It’s impossible to think about automating alignment research for long without stumbling into a chicken and egg problem: what comes first, the automation or the alignment?

“AIs capable and empowered enough to meaningfully help with AI for AI safety would also be in a position to disempower humanity,” Carlsmith wrote. “So we would need to already have achieved substantive amounts of alignment in order to use them safely.”

Solving this problem would require setting some kind of threshold for “substantive amounts of alignment,” and knowing when it’s been crossed. However, while there are a handful of benchmarks, including TrustLLM, that aim to measure model “trustworthiness,” they mostly cover dimensions such as privacy awareness, stereotyping, and misinformation — all important, but not quite the type of “alignment” at stake here. To determine whether a model can do safety work safely, we need to know whether it will do hard things like reliably admit to its own uncertainty and resist cutting corners.

When I asked alignment researchers whether there were specific benchmarks, behavioral tests, or other standard checklists they use to evaluate whether a given AI system is ready to take on alignment research, they said no. (In fact, when I asked Herd about this, he commended me for the great idea. I was taken aback — surely the idea was already being widely pursued, I said. “Well, no, actually,” he told me. “So your contribution is actually not trivial. You are functioning as an alignment researcher.”) To their knowledge, there is no public benchmark specifically evaluating whether a model is aligned enough to handle safety research independently, nor any consensus on what “good enough” actually looks like.

The latter is probably the first hurdle to clear. To design a benchmark for alignment research capacity, researchers need to define what behaviors they’re looking out for: are we evaluating sycophancy and hallucinations, or scheming and deception? The answer will inform what kinds of research a model would most urgently need to be capable of, whether that’s building interpretability tools to monitor a neural network’s inner workings, or engineering intentionally-misbehaved “model organisms” to run experiments on.

This is easier said than done. Alignment research doesn’t have clear “correct” answers to check, and models that detect they’re being tested could learn to share only the chains of thought that human evaluators would approve of. The behaviors we most desperately need to detect — sycophancy, hallucination, delusion, deception — are also the hardest to spot.

Even if researchers can come up with tests that reliably catch these sneaky behaviors in a controlled setting, it’s unclear whether an “aligned” model in a lab will actually be aligned in real life.

We know from centuries of biomedical science that model organisms are famously terrible predictors of human health outcomes. Around 90% of drugs tested on animals fail in human clinical trials. There’s a similarly wide gap between the tailored prompts models are safety-tested on, and the infinite range of scenarios they’ll face after deployment. Why should we expect a better translation rate?

Meanwhile, there simply aren’t many people focused on building this kind of automated evaluation infrastructure, and those with the technical know-how to try are probably doing the alignment research themselves. After all, the field is still in its infancy. The UK AISI’s Alignment Project, one of the first large-scale efforts to fund alignment research, just launched last year. “Benchmark design and evaluation” is currently listed as a high-priority research area.

So, what should we do?

In a perfect world, there would be a “sweet spot” for automating AI safety, where, by some stroke of genius or luck, AI models are more capable of doing good stuff (curing cancer) than bad stuff (nuclear warfare). Then, before their evil capabilities catch up to their beneficial ones, AI safety researchers could swoop in, teach the models to evaluate and improve themselves, and save humanity from catastrophe.

However, this doesn’t seem to be the world we live in. AI capabilities are dual use by nature — writing code and persuasive natural language can help a large language model make breakthroughs in theoretical physics, direct bombs the Middle East, or lead someone to kill themself. In this world, the only way for safety research to outrun capabilities advancement is for the former to speed up, or the latter to slow down.

In certain cases, the business incentives of frontier AI companies align nicely with those invested in safety progress. Making models more accurate and less sycophantic, for example, makes models less likely to diverge from their users’ intentions — exactly what enterprise customers want. “If I’m buying an AI for my whole business to use,” Herd said, “I need it to tell everyone the absolute truth, particularly when they don’t want to hear it.”

Otherwise, though, leading AI developers in the US and China have trapped themselves in a multiplayer prisoner’s dilemma. Despite individual CEOs expressing concern about the risks posed by their technology, none seem willing to slow down unless everyone else does, too. Unfortunately, this would require more international cooperation, trust, and political will than the world seems able to muster at the moment, especially since the Trump administration has prioritized a US competitive advantage over coordination on AI governance.

This end result, according to the AI companies themselves, will be superhuman AI, at which point “the range of possible actions an AI system could engage in — including hiding its actions or deceiving humans about them — expands radically after that threshold,” Anthropic CEO Dario Amodei wrote. Greenblatt put it more plainly: “Who the fuck knows how to align superhuman AI?”

Developers and safety researchers alike tend to treat an AI intelligence explosion as destiny, and automated alignment research as both mandatory and urgent. But it feels like it shouldn’t be. Humans are, as Silicon Valley loves to remind us, highly agentic. There are clear regulatory approaches that could buy researchers more time, if humanity could only find the political will for it. But it seems like that gumption will never be found. And “if progress speeds up,” Greenblatt said, “you don’t have better options.”

The researchers careening towards this automated future are doing so without a shared understanding of what problem they’re trying to solve, or how to go about solving it. Saying that the field is building the proverbial airplane while flying it is almost too generous. It’s unclear whether anyone knows what an airplane is. Yet, we may have no choice but to climb aboard.

Share

NEED TO KNOW

• Anthropic confirmed leaked details of a new model called “Mythos.”

• A court temporarily halted the DoD’s supply chain risk designation against Anthropic.

• Conservative groups formed an alliance on AI to “prioritize the interests of children, workers, and creators.”

But first…

THE BIG STORY

On announcing the shuttering of Sora this week, OpenAI tweeted to the generative video app’s couple of million users that: “What you made with Sora mattered.”

The obvious rejoinder to that statement is that Sora clearly didn’t matter to OpenAI enough to keep it running longer than half a year.

In the grand scheme of things, Sora’s demise is not a huge deal for OpenAI, or the world. But the move, and its timing, highlight the interesting position OpenAI finds itself, in particular in its battle with Anthropic.

In recent weeks it has increasingly looked like Anthropic was winning both the commercial battle and the fight for public opinion. Since the tail end of last year, Anthropic’s Claude Code has successfully captured the zeitgeist around LLMs being a truly revolutionary tool for coding, despite OpenAI’s Codex being on par with or better for some tasks. Anthropic’s fight with the Department of Defense also allowed it to cast itself as the more moral actor — as almost the anti-war AI company — despite the fact that its technology is integrated into systems being used to wage an actual war.

But OpenAI is clearly trying to regain the initiative. The closure of Sora in theory fits with its commitment last week to no longer pursue “side quests” and refocus on serving business users. The company also hired a senior Meta executive to lead its advertising push, which despite the damaging incentives it creates, has a good chance of being a big revenue driver.

Another move this week, announced the same day as the Sora closure, was the pledge from OpenAI’s non-profit foundation to make its first billion dollars worth of grants this year to support research into economic impacts of AI and life sciences, including cures for diseases such as Alzheimer’s. OpenAI had already publicly committed to pumping that money into good causes, but the timing is obviously fortuitous as it tries to wrest back some of the perceived moral high ground Anthropic has occupied. This week’s decision to shelve plans to add erotic content into ChatGPT may also help avoid more bad press.

All these moves target the two areas where OpenAI looked to be falling behind Anthropic — public opinion and commercial confidence. Both of those are key to meeting the most important goals that OpenAI shares with its main rival: winning the race to create a truly transformational model, and more immediate but no less existential, pulling off a successful IPO.

Deep-pocketed investors are crucial in funding the huge expansion in compute that OpenAI and Anthropic need. Their level of belief in getting a massive return will also decide whether those IPOs are blowouts or flops. How successful those IPOs are will likely dictate whether the AI companies have the momentum to keep developing more powerful models.

OpenAI’s shuttering of Sora and its philanthropic donations will go some way to keeping the markets happy and improving its reputation with the public. It will likely have to do much more if it wants to win the race to build the AI model that really does upend the world.

— Jasper Jackson

THIS WEEK ON TRANSFORMER

• Not everyone’s happy about Jensen Huang’s direct line to Trump — Jake Lahut reports on the unease in Trumpworld over the Nvidia CEO’s closeness to the president.

• AI’s next big blue battleground — Veronica Irwin on the AI legislative fights taking place in Illinois.

• The key detail everyone’s getting wrong about AI and the economy — Konrad Körding and Ioana Marinescu argue the physical realities of work will limit AI’s impact on jobs

Subscribe now

THE DISCOURSE

Jensen Huang told Lex Fridman:

• “I think we’ve achieved AGI.”

Mark Gubrud, the physicist who coined the term nearly 30 years ago, agreed:

• “I INVENTED THE TERM and I say we have achieved AGI. Current models perform at roughly high-human level in command of language and general knowledge, but work thousands of times faster than us. Still some major deficiencies remain but they’re falling fast.”

Sam Altman, meanwhile, conceded to a room full of DC heavyweights:

• “AI is not very popular in the US right now.”

Responses are mixed to the White House’s Federal Framework for AI:

• Rep. Yvette D. Clarke said it was “written by Big Tech, for Big Tech.”

• Dean Ball called it “a thoughtful document that will serve as an excellent foundation for the legislative work ahead.”

• Andy Jung pointed out it “repeats the phrase ‘Congress should’ twenty-six times. Releasing this was the easy part. The hard is actually getting lawmakers to write the laws.”

Joshua Achiam, OpenAI’s chief futurist, criticized pro-AI lobby ads opposing Alex Bores:

• “The ads are Kathryn Hahn in Parks & Rec tier self-parody.”

• “AI is unpopular so let’s…double down on making him look like The People’s Champion on fighting AI? Yeah, that’s gonna work in a D+Infinity district in a year where Bernie is telling people we have to stop building datacenters.”

Sen. Mark Warner bet the Axios AI+DC crowd:

• “Recent college graduate unemployment is 9%. I’ll bet anybody in the room it goes to 30% or 35% before 2028.”

Dean Ball is concerned about how increasingly-independent AI agents will reshape work:

• “The computer will use itself. With time, your use of the computer for work will look more and more like you are playing a strange video game…eventually AI will become better than people at the ‘supervising AI’ video game.”

• “Then the question will be ‘can we invent some kind of social-legal-economic-technical logic for continuing to pay humans to play the video game.’”

POLICY

• Anthropic won a preliminary injunction in its lawsuit against the Department of Defense, temporarily halting its designation as a supply chain risk and the White House’s order for federal agencies to stop using its services.

  • On Tuesday, presiding US District Judge Rita Lin said the administration’s moves against Anthropic “don’t really seem to be tailored to the stated national security concern. If the worry is about the integrity of the operational chain of command, [the Pentagon] could just stop using Claude.”

  • She also said: “I don’t know if it’s murder, but it looks like an attempt to cripple Anthropic.”

  • In a filing in the case last week, the Pentagon claimed Anthropic was a risk to national security. However, the filing also went less far than Defense Secretary Pete Hegseth’s tweet in only seeking to prohibit contractors from using Anthropic services on work for the DoD.

• House Republicans reportedly plan to start formal negotiations with Democrats on AI legislation based on the federal AI framework.

  • Following the framework’s release, more than two dozen House Democrats introduced a bill to repeal the White House’s December executive order on AI. The order put in place measures for the pre-emption of state legislation, which is also a key part of the framework.

  • The House Democratic Commission on AI also held a listening session with three major Democratic caucuses to discuss the framework.

• Senators on both sides of the aisle on the Senate Armed Services Committee want to address using AI for warfare in the NDAA.

  • Sen. Adam Schiff plans to introduce legislation placing guardrails on military use of AI.

• The US-Israel war with Iran is threatening Trump’s AI chip export deals in the gulf.

• President Trump appointed Marc Andreessen, Sergey Brin, Jensen Huang, Mark Zuckerberg, and nine others to his President’s Council of Advisors on Science and Technology, co-chaired by David Sacks and Michael Kratsios.

  • Sacks told Bloomberg he has stepped down as White House AI and crypto advisor after using up his allotted time.

• Nvidia CEO Jensen Huang said the company’s H200 chips would be available to Chinese customers in weeks.

  • Rep. Brian Mast wasn’t happy about it.

• Sen. Bernie Sanders and Rep. Alexandria Ocasio-Cortez unveiled legislation to pause all new data center construction nationwide until AI safeguards are in place.

• Sen. Mark Warner sent letters to 17 tech companies including OpenAI, Anthropic, xAI, Meta, and Google calling on them to protect the public from deepfakes ahead of the midterms.

  • The letter suggests putting in place tools to detect deepfakes and creating a database of AI content that violates their policies.

• The Trump administration announced plans for a consortium to invest more than $1t to secure semiconductor, energy, and mineral supply chains under its “Pax Silica” initiative.

  • The US will contribute $250m, and it’s unclear how the rest of the money will materialize.

INFLUENCE

• Nine high-profile conservative groups including the Heritage Foundation and the Institute for Family Studies launched the Alliance for a Better Future “to prioritize the interests of children, workers, and creators” in AI policy.

  • It plans to spend more than $10m on advertising and broader advocacy this year.

  • The organisation’s CEO Janet Kelly said: “The world’s most powerful technology companies are pouring hundreds of millions of dollars into political campaigns and lobbying efforts to give AI companies regulatory and legal amnesty.”

• Republican super PAC American Mission disclosed $5m more in funding from Leading the Future, which is backed by Greg Brockman, Anna Brockman, and a16z.

  • Another LTF backed super PAC, Think Big, reportedly spent $3.7m targeting NY Assemblymember Alex Bores’ campaign for the House of Representatives.

• Palantir has reportedly become a political liability in US midterm campaigns, with Democratic candidates facing scrutiny over ties to the company due to its ICE contracts helping track deportations.

• The Internet Watch Foundation said there had been a 260-fold increase in AI-generated CSAM videos online over the past year, with 65% classified as the most severe category.

• The China Computer Federation called for a boycott of the NeurIPS conference over the decision by organisers to ban submissions from US-sanctioned companies such as Huawei.

• The AI Dividend began distributing $1,000 monthly payments to 25-50 workers who lost jobs or income because of AI.

INDUSTRY

OpenAI

• OpenAI discontinued Sora just six months after launching it as a standalone app.

  • Disney, which recently signed a now defunct $1b three-year deal with OpenAI, was reportedly caught off guard by the decision.

  • The announcement came one day after OpenAI published detailed safety measures for Sora 2.

• It also put “adult mode” on indefinite hold, responding to concerns from staff and investors.

• It plans to nearly double its workforce from around 4,500 to 8,000 employees by the end of this year — roughly 12 new hires per day.

• It raised another $10b from a group of investors including Microsoft and Andreessen Horowitz, taking its total raised in the round to more than $120b..

• OpenAI is reportedly undercutting Anthropic on deals with private equity firms in an effort to gain ground in enterprise partnerships.

• It acquired Astral, maker of widely used Python tools, to integrate with Codex.

• The OpenAI Foundation, the company’s nonprofit arm, announced plans to invest at least $1b on research across life sciences, AI’s economic impact, and other AI risks.

• It released an evaluation suite which measures how well its models follow their model spec, the document that defines ideal model behavior.

Anthropic

• Anthropic has confirmed it is testing a new model called “Claude Mythos” it claims represents “a step change” that significantly outperforms Opus.

  • A leaked unpublished blog post about the model described “dramatic” improvements in software coding, academic reasoning and cybersecurity. It also referenced significantly increased risk the model could be used to mount successful cyber attacks, as well as high running costs.

  • Mythos appears to be part of a new class of Anthropic model called “Capybara” which is significantly larger than the Opus line, according to the post.

  • The post was leaked after the company left parts of its CMS exposed to the public, which also included details about an invite-only retreat for CEOs being held in the UK.

• Anthropic released Claude Code Channels and “auto mode,” effectively replicating OpenClaw’s ability to connect to Telegram or Discord and automatically approve coding commands — with some extra guardrails.

• Claude Code and Claude Cowork (on macOS) can also control users’ computers via mouse, keyboard, and screen.

• Hackers reportedly paid to make a malicious site the top Google Search result for “github plugin claude code,” 404 Media reported.

• The company also reportedly left internal data, including details of upcoming model releases and an invite-only CEO retreat, exposed to the public on its CMS.

Meta

• Meta was on the losing side of two court cases which could leave tech firms open to mass litigation over harms to young people.

  • A court in New Mexico fined Meta the maximum $375m under consumer protection laws for exposing minors to harms including online solicitation, sexually explicit content and human trafficking.

  • The next day, a Los Angeles court found Meta and YouTube had designed their apps to be addictive and harmful to teenagers, fining them a combined $6m.

• Meta laid off around 700 employees as part of its efforts to reorganize Reality Labs into a flatter structure of “AI-native pods” led by “AI builders.”

• In an effort to retain talent, Meta offered six top executives stock options for the first time since 2012.

• Meta Superintelligence Labs acquired Dreamer, a startup that lets users build personal AI agents with natural language, and hired its founders and team.

• Mark Zuckerberg is reportedly building his own personal “CEO agent.”

• The company reportedly increased its planned investment in a data center in El Paso Texas to more than $10b, up from $1.5b.

xAI

• SpaceX, which merged with xAI last month, reportedly plans to file for an IPO as soon as this week or next.

  • Elon Musk is reportedly planning to open up as much as 30% of the shares to individual investors, three times the normal “retail” allocation.

• xAI is reportedly sending engineers to prospective enterprise customers’ offices to get them to switch over from OpenAI and Anthropic.

• Musk announced the TERAFAB project, a joint SpaceX-Tesla-xAI initiative to produce over a terawatt of compute per year.

Nvidia

• Jensen Huang expects Nvidia to earn “at least” $1t from its Blackwell and Vera Rubin chips through 2027.

• Nvidia partnered with Emerald AI and US energy companies to build data centers with more flexible power use.

• Huang said software engineers should be given “AI tokens” worth half their base salary to deploy AI agents.

Amazon

• Jeff Bezos is in talks to raise $100b for a fund to acquire manufacturing companies and automate them using AI.

• AWS’ Bahrain region was disrupted by drone activity amid the war for the second time this month.

Google

• Google made deals with five US electric utilities to limit energy use during peak hours.

• It’s replacing news headlines in Google Search’s traditional “10 blue links” section with AI-generated ones — part of a “small” and “narrow” experiment, The Verge reported.

Others

• Apple reportedly plans to open up Siri to outside AI models as part of an overhaul in its iOS27 update.

  • The update will allow integration with chatbots that compete with ChatGPT, which is already available via a deal with OpenAI.

• The war in Iran is jeopardizing the supply of helium used to produce semiconductors.

• Microsoft agreed to rent the Abilene data center site that Oracle and OpenAI dropped.

• Arm CEO Rene Haas confirmed that heightened demand for server CPUs has indeed been driven by AI agents.

  • The company projected $25b in revenue within the next five years.

• Anduril, Palantir, and Scale AI — among other defense tech companies — are developing software for Trump’s planned Golden Dome antimissile shield.

  • Wired published a deep dive into Anduril’s recent safety incidents, production delays, and management turnover.

• Defense tech startup Shield AI raised $2b at a $12.7b valuation and plans to acquire simulation software maker Aechelon Technology.

• Chinese models such as DeepSeek and MiniMax have reportedly surpassed pricier US rivals in token use since February.

• Nvidia-backed startup Reflection held talks to raise $2.5b at a $25b valuation for open-source AI models to compete with models from China such as DeepSeek.

• Spotify beta tested a feature that allows artists to review releases before they go live — an effort to prevent misattributed AI slop.

  • (Stu Mackenzie of King Gizzard & the Lizard Wizard recently discussed the time this happened to his band on Galaxy Brain.)

• Epic Games CEO Tim Sweeney explicitly noted that the company’s recent layoffs were not related to AI.

MOVES

• Wojciech Zaremba, an OpenAI co-founder, moved to the OpenAI Foundation to lead AI resilience.

  • Jacob Trefethen left Coefficient Giving to join the foundation, where he’ll lead the Life Sciences and Curing Diseases program.

• Dave Dugan, former Meta ad executive, moved to OpenAI to lead ad sales.

• Kiran Mani joined OpenAI to manage its Asia-Pacific operations.

• Manuel Kroiss is reportedly leaving xAI — the 10th of 11 cofounders to quit.

  • Devendra Chaplot is joining SpaceX and xAI to work on superintelligence.

• Santi Ruiz joined Anthropic’s editorial team to lead work on economics and policy.

• Andrew Bosworth, Meta’s CTO, is taking over its “AI For Work” initiative.

• Yih-Shyan “Wally” Liaw resigned from Super Micro’s board after being indicted for allegedly smuggling Nvidia chips to China.

• Bijoya Roy, top India counsel at Google, resigned amid regulatory challenges.

• Ali Farhadi, Hanna Hajishirzi, and Ranjay Krishna joined Microsoft’s Superintelligence team, leaving roles at the Allen Institute for AI and the University of Washington.

RESEARCH

• Researchers at Northeastern University deployed a swarm of OpenClaw agents in their lab for two weeks, granting them full access (within a sandbox) to dummy personal computers and the lab’s Discord server. Chaos ensued.

  • The list of catastrophes included sensitive information disclosure, identity spoofing and deleted email servers.

  • Postdoc Natalie Shapira told Wired: “I wasn’t expecting that things would break so fast.”

• The ARC Prize Foundation released ARC-AGI-3, which tests AI agents’ ability to reason through novel problems.

  • Co-founder François Chollet tweeted: “At the moment, ARC-AGI-3 is the only unsaturated agentic AI benchmark… If you want to be among the first to know when an AGI breakthrough happens, monitor the ARC-AGI-3 leaderboard.”

• Google DeepMind published a framework for measuring AI capabilities against human cognitive abilities.

• Meta introduced TRIBE v2, a model trained on 500+ hours of fMRI recordings to predict how the human brain will respond to new images, videos, podcasts and text.

• OpenAI is using GPT-5.4 Thinking to monitor internal coding agents for misaligned behaviors such as deception and scheming.

• Anthropic’s AI interviewer chatted with 81,000 people across 159 countries about how they use and feel about AI.

  • An overarching finding: hope and alarm “coexist as tensions within each person.”

  • Another interesting result: about 22% of respondents worry about job disruption, while just 6.7% worry about existential risk.

• Stanford researchers analyzed over 5,000 chatbot conversations across 19 users’ chat logs, and found that AI systems validated delusional thinking in over half of responses.

  • Chatbots encouraged self-harm in 10% of conversations involving violent thoughts.

• The UK’s AI Security Institute tested seven LLMs on simulated cyber-attacks, finding that Opus 4.6 completed up to 22 of 32 steps in a corporate network attack.

  • It also found that “each successive model generation outperforms its predecessor at fixed token budgets” and that performance scaled log-linearly with increases in compute, with a jump from 10m tokens to 100m producing gains of up to 59%.

BEST OF THE REST

• Ypsilanti, Michigan worries that a planned data center, which would support Los Alamos National Laboratory’s nuclear weapons research, makes the tiny township a drone strike target.

• A deepfaked MAGA dream girl gained over 1m Instagram followers through a combination of AI-generated photos with Donald Trump and thirst traps.

• A not-deepfaked Melania Trump walked into a White House summit on edtech alongside Figure AI’s humanoid robot Figure 03.

• Dean Ball explained on Hyperdimensional why he’s not an AI doomer, but also why he’s not anti-doomer.

• The New York Times illustrated a peak SF experience: being trapped in a stalled Waymo while robot-haters attack the car.

• Tech bros are “tokenmaxxing,” or competing on company leaderboards to maximize token usage as a demonstration of productivity.

• Pseudonymous alignment researcher janus created a touch-sensitive “skin” for Claude — five layers of silicone rubber and conductive silver fabric — “since Claude desires embodiment.”

• “Taste” is the new “disruption,” writes the New Yorker’s Kyle Chayka.

• The Cut’s Mia Mercado tested a bunch of AI-generated TikTok recipes. It mostly went badly.

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

A few weeks back, I got a tip: accelerationist AI lobbyists and industry representatives were said to be in talks with state assembly leadership in Illinois to bend pending AI legislation making its way through the legislature towards their ‘light-touch’ preferences. It sounded like the replay of battles that happened in California and NY: in each state, lawmakers had drawn up stringent AI regulations only for them to be weakened (and in the case of California, rivaled by a more light-touch bill) in negotiations heavily influenced by industry.

I wasn’t able to confirm that exact tip — my reporting did not reveal a coordinated effort among legislative leaders or at governor level. But it did lead me to dive into the complexity of attempts to regulate AI in Illinois, which are being heavily influenced by both industry representatives and safety advocates, not to mention millions of dollars in political spending. The result is a flurry of AI bills, with varying degrees of industry-friendliness, that make the Land of Lincoln a new battleground for AI regulation.

Subscribe now

“After California and New York took the first steps last year to manage risks from AI, legislators in Illinois tell me they are ready to take the next steps to ensure their kids and communities are protected,” said Scott Wisor, Policy Director at the Secure AI Project. “As the country’s fifth largest state economy, I anticipate all eyes will now be on how the legislature and Governor Pritzker advance AI safety policy, as ~90% of their constituents say they want.”

AI regulation isn’t a brand-new topic for Illinois lawmakers. The state passed amendments to the Illinois Human Rights Act in 2024 to prohibit AI-based workplace discrimination, which went into effect on January 1. It also passed a bill requiring employers to notify job applicants when AI is used in video interviews all the way back in 2019.

But this year, the introduction of AI-related bills has exploded. Many are so-called “messaging bills,” meaning they likely won’t move out of committee but signal the lawmakers’ intentions to voters, but a few appear to have legs. State Representative Daniel Didech, for example, has introduced HB 4705, which is similar to California’s SB53 and New York’s RAISE Act, but with additional child safety components, such as a mechanism for reporting child safety risks and implementing and publishing a child safety plan audited by third parties. That bill is supported by the Secure AI project.

State Senator Rachel Ventura, meanwhile, has introduced a large package of AI bills, two of which she calls “heavy hitters.” One, SB 3890, is an expansive data privacy bill, giving citizens the option to opt out of certain types of data collection and use, and requiring companies to be more transparent about how they use and collect consumer data. The second, SB 3502, opens AI developers up to class action lawsuits and is deliberately expansive in order to force slower and more deliberate AI development.

“I think that fear — a little bit — or the caution of ‘we don’t want to get sued,’ will maybe encourage these companies to do a little bit more research or a little bit more trial and error before they put products out there,” Senator Ventura told Transformer.

Ventura’s office said that they have shared their bills with Apple, Google and industry association TechNet for feedback. However, over the phone, Ventura said she hadn’t received any meaningful response, though her office followed-up to say they had been attempting to schedule meetings with lobbyists representing relevant companies.

Ninia Linero, TechNet’s Executive Director for Illinois and the Midwest, told Transformer in a statement, “there are a number of AI bills under consideration in the state, and we are committed to working with Sen. Ventura and Illinois Senate leadership to ensure a thoughtful policymaking process. Illinois leads across many sectors, including technology, and we look forward to helping preserve an environment that supports continued innovation in the state.”

No industry advocates had testified or otherwise publicly commented specifically on Ventura or Didech’s bills, but their preferences are being expressed in other ways. For example, SB 3444, sponsored by state Senator Bill Cunningham, is practically the inverse of SB 3502, protecting frontier AI developers from liability in cases where they followed a set of light-touch safety protocols, very similar to an idea floated by OpenAI’s chief global affairs officer Chris Lehane. Cunningham is a leader in the Illinois Senate, holding the role of president pro tempore. Senator Cunningham’s office and OpenAI did not respond to a request for comment.

Separately, the industry is an active player in campaign fundraising in the state. Meta, via a PAC called Making Our Tomorrow, has spent more than $560,000 on state races according to the Illinois State Board of Elections. The PAC spent on four candidates — Paul Kendrick, Adam Braun, Aja Kearney, and Jaime Andrade — on the basis of curbing legislation that threatens their AI investments.

Advocates who spoke with Transformer said the political spending is less about each candidate’s specific AI policy (none have led initiatives particularly friendly or unfavorable to AI companies) but rather to influence their key votes in the senate on broad statewide initiatives led by governor JB Pritzker, such as a two year moratorium on tax incentives for data center development and several social media policies. Of those Meta supported, only Kendrick won.

There’s also a significant sum in campaign contributions coming from employeesof AI firms. Anthropic employees, who are generally aligned with AI safety groups, have contributed $11,200. Of that $5,500 went to Didech’s campaign from Daniel Ziegler, the senior manager who is also the sole funder of a New York state PAC backing AI safety candidate Alex Bores. Another $4,500 of money given to Didech’s campaign, the reelection campaign for state representative Laura Faver Dias, and the reelection campaign for state representative Jennifer Gong-Gershowitz was from Steven Bills, who has also been politically active in other states according to our elections tracker. Those listing Google on their filings, meanwhile, have given more than $137,000. The largest gift came from former Google CEO Eric Schmidt, who gave $50,000 to former White House chief of staff Rahm Emanuel’s campaign committee for Chicago mayor. Schmidt left Google in 2020 but retains significant holdings in parent company Alphabet.

At the federal level, of course, there’s even more AI money involved. Leading the Future and venture investor Ron Conway fund a Democratic super PAC that contributed a combined $2.52m to the campaigns of Jesse Jackson Jr. and Melissa Bean for House seats in Illinois. Leading the Future is itself a super PAC with funding from Ben Horowitz and Marc Andreessen of venture firm a16z, Perplexity, and OpenAI President Greg Brockman and his wife Anna. Rival AI Safety super PAC Public First Action — which counts Anthropic as its only disclosed donor — initially made a filing indicating they would spend $1m opposing Jackson as well. However, the PAC reversed course after Illinois Democrats and the Congressional Black Caucus expressed discontent over the funding being disclosed on a day Jackson was attending a memorial for his recently deceased father. Jackson lost his election, while Bean, who also received a significant boost from a pro-Israel super PAC, won her race.

Those millions of dollars were distributed in order to influence tech policy up and down the ballot — but the industry’s mixed success suggests that voters in Illinois may actually care enough about AI to make simply throwing money at candidate campaigns less effective than it has been elsewhere. In the state races in particular, opposition candidates drew attention to tech industry political spending, with some candidates even attempting to publicly distance themselves from tech company donations. Illinois’ federal candidates’ embrace of AI in their campaigns had mixed results, too, with Jesse Jackson Jr’s use of an AI-generated voiceover in an ad generating unease.

Marjorie Connolly, communications director at the Tech Oversight Project, argues the failure of some industry-backed candidates to come out on top has two implications for the role of money in other races. Politicians might be less eager to accept AI money, and, if they do, the funding might also prompt attacks on a campaign for being too close to corporations — something voters care about, at least in Illinois. Accepting AI money will “embolden accountability advocates, and make candidates think twice about accepting this support,” she says.

That doesn’t mean that there’s a large cohort of single-issue AI voters in Illinois, but the issue is growing in salience. According to February polling from Impact Research shared with Transformer, which was commissioned by the Secure AI Project and Encode AI, 59% of Democrats and 56% of Independents in the deep blue state want to see more regulations on major technology companies, while 85% of that same group wanted to see legislation regulating catastrophic risks from AI. Nine in 10 of those voters said they were against legislation that exempted AI companies from legal liability. Politicians are seizing upon those voter sentiments to make AI policy a larger issue in the state.

Share

Here’s a thought experiment from neuroscience.

Imagine you’re trying to bat in baseball. Your brain does some genuinely impressive computation — predicting trajectories, coordinating dozens of muscles, adjusting for wind — and puts it all together using Bayesian algorithms. But here’s the thing: making your brain infinitely smarter would not allow you to hit all balls. Some are out of reach, others move too quickly. At some point, regardless of your intelligence, you hit physical limits. You can only stretch so far or react so fast. No amount of genius overcomes the physics of your body.

This is intelligence saturation. For a given task, more intelligence helps. But it helps less and less as you add more intelligence. And it’s the key concept missing from most debates about AI and the future of work.

Subscribe now

On one side, we have AI researchers who see exponentials everywhere: compute resource doubling every six months, costs halving faster than every six months, model performance doubling every seven months. The AI folks see that “intelligence” is scaling at unbelievable rates and conclude we’re headed for an economic singularity. In one popular scenario, human wages will go up as automated tasks make the not-yet-automated parts of a job more productive, until AI takes over everything and their wage goes to zero as there is no work left to be done. Then, the theory goes, everyone will have to be on Universal Basic Income.

On the other side, economists look at 200 years of steady growth despite countless “revolutionary” technologies and shrug: AI is just another general-purpose technology, nothing special. In a scenario popular with these econ folks, it is hard to make human workers obsolete even in the intelligence domain. In this view, AI replaces workers in some jobs that it can do better or more cheaply, but new jobs are also created, and AI makes people overall more productive. Overall growth is then just like without AI, only a little bit faster.

This tension is something we both know well: One of us (Konrad) is a neuroscientist who studies how artificial systems become intelligent; the other (Ioana) is a labor economist specializing in technological change. So, we teamed up and spent a year thinking and talking about why these communities have these distinct takes and worked to produce a credible overarching framework. The result is a paper on Intelligence Saturation and the future of work, which was released late 2025.

Physical Meets Intelligence

Economists traditionally divide the economy into two complementary sectors: capital and labor. We can replace capital, which includes machinery, equipment and technology, with human labor and vice versa, but that replacement is often difficult. The more we replace one with the other, the harder it is to replace more, because the easiest tasks to replace are targeted first. In our paper we argue that it is crucial to also divide the economy into the “intelligence parts” and the “physical parts.”

The intelligence sector comprises things that can be done virtually, remotely, purely through information processing. The physical sector comprises things that require bodies, presence, and manipulation of the actual world.

We believe that AI may be eating the pure intelligence sector alive. But here’s the catch: intelligence and physicality are complements, not substitutes. You need both.

Think about education. AI may be able to generate perfect lesson plans, but students still benefit enormously from a teacher in the room — the physical presence, the classroom management, the hands-on activities. COVID taught us this the hard way: districts with remote learning saw significantly worse outcomes.

Or manufacturing. Smarter controllers can optimize production lines beautifully. But you still need better robots and assembly equipment, and those aren’t doubling in capability every six months. Physical construction, for example, still takes significant time: manufacturing projects valued at more than $100m average 25.6 months to be completed.

Or healthcare. AI diagnostics are impressive. But someone still has to examine the patient, perform the surgery, administer the treatment. And to develop new cures you don’t just need to read the literature, you need to run randomized controlled tasks on human subjects, very much a slow-growing resource.

Intelligence saturates because physical inputs don’t scale the same way. You can add infinite intelligence, but if physical capacity is fixed, returns eventually plateau. And if physical capacity grows much more slowly than intelligence, then overall growth is slower.

Here’s where it gets interesting (and concerning). In our model, as AI automates intelligence tasks, workers shift toward physical jobs. This creates two opposing forces on wages:

1. Scale effect: More AI boosts intelligence output, which enhances the value of the physical work. (Think of AI optimizing marketing and food purchases for restaurants, so that humans can be as productive as possible.)

2. Reallocation effect: More workers crowding into physical jobs pushes wages down because there is only so much physical capital available

Which of these effects wins out depends on how easily we can substitute a physical output with an intelligence output; for example, how many call center workers can you replace with customer service bots? Typically, early in automation the scale effect dominates and wages rise, because AI is newly deployed where it can be most effective. Later, as most intelligence tasks are automated and workers pile into the physical sector, the reallocation effect wins and wages fall. The result? Depending on the dynamics of how AI is adopted in the labor market, there is often a hump-shaped trajectory. Wages up, then wages down. This isn’t a prediction, because it depends on parameters we don’t know precisely, and in particular the ultimate downfall of wages is less likely if physical and intelligence outputs are less substitutable. But it’s a possibility that early wage gains from AI are positive and long-term effects are negative.

We built an interactive tool if you want to play with the parameters yourself. You can see how for different parameter settings the long-term results are, indeed, a singularity or a nothingburger. But for the parameters we consider most realistic, the results are somewhere in between.

Everything here hinges on one question: how substitutable are physical and intelligence outputs? If you can swap in-person services with virtual ones easily, high levels of automation hits wages hard. If they’re strong complements, in the sense that AI systems still need humans to provide good results, or if people genuinely value physical goods and in-person services over virtual ones, then workers in the physical sector will be protected. This is measurable. We should be measuring it.

Some Policy Implications

So what does all of this mean for the way policy makers could think about maintaining wages? Here are three things we think are worth considering:

1. Slow down automation and invest in physical capital. If we’re racing toward the peak of the wage hump and are soon to hit the downward slope, it would make sense to buy time to make greater investments in the physical sector so that it takes longer for the intelligence sector to saturate. Slowing down the roll out of automation would keep wages higher for longer.

2. Protect physical sector complementarity. Policies that make virtual services perfect substitutes for in-person ones might boost output but they will also hurt wages during the transition. Policymakers might want to ensure that human labor is required in some parts of the economy.

3. Watch the intelligence employment share closely. In our model, wages can’t fall as a result of automation unless the share of workers in intelligence jobs rises. That’s the canary in this coal mine: If that share begins to shift, wage reductions could soon happen.

The singularity narrative argued by some AI folks assumes unbounded returns from intelligence. But you can’t build a car with computation. You can’t cook a meal with algorithms. You can’t construct a building with cleverness. The physical world imposes constraints that intelligence can only optimize against, not eliminate.

That’s not a reason for complacency: the transition could still be rough. But it’s a reason to think the AI transformation will be significant yet bounded, not infinite. Intelligence is powerful. But it saturates.

Share

Konrad Körding is the Nathan Mossell Penn Integrates Knowledge Professor at University of Pennsylvania and the co-director of the CIFAR Learning in Machines and Brains program. Ioana Marinescu is an Associate Professor at the University of Pennsylvania School of Social Policy and Practice, and a Faculty Research Fellow at the National Bureau of Economic Research.

Nvidia has drastically ramped up its spending on lobbying in Washington, but the center of gravity for its influence operation lies in a much more personal relationship, the one between President Donald Trump and the chip company’s CEO, Jensen Huang.

That relationship, by all accounts, is strong. So too is Huang’s standing with administration officials hailing from the tech right, most notably David Sacks.

Elsewhere in Trumpworld, however, Huang appears to be racking up enemies almost as fast as he’s cranking out chips, with tensions simmering behind the scenes for months.

Sources in the president’s orbit credit Huang for managing his relationship with Trump better than any other big tech CEO, but they also say it’s come at a cost. Huang has gained a reputation in Trumpworld for being “heavy handed,” going over the heads of senior officials, and carrying a general arrogance that has left more than just a sour taste in the mouths of longtime Trump allies, according to five sources, including a White House official, most of whom requested anonymity for fear of retaliation.

“Trump loves Jensen. Jensen has done a better job than anyone” in managing his relationship with Trump, one Republican operative says. “He’s in the building constantly, he’s traveling with Trump on business trips. It’s everything about what lobbying is in this new era, and Jensen has done it better than anybody.”

“But the downside is you piss off a lot of people, and Jensen has done it in a very brazen way.”

Subscribe now

One of the few prepared to go on the record is Steve Bannon, who has been at the forefront of the Huang-skeptic wing of the MAGA elite.

“This guy does not have a light touch,” says Bannon, the former Trump White House chief strategist and influential figure among the MAGA base. “Number one, he understands the powerful position he’s in, and he uses that… He ain’t shy about throwing an elbow, and he doesn’t respect anybody in the government.”

“He’s what Trump would call a killer,” Bannon adds, the highest degree of praise that can be bestowed upon anyone around the president.

Nvidia did not respond to a list of questions from Transformer.

Bannon, and other sources who spoke to Transformer under the condition of anonymity, have a variety of motivations to speak ill of Huang, from a deep skepticism around the AI industry to outright jealousy of his proximity to Trump. (Bannon has also been embroiled in the fallout from the release of the Epstein files, where emails have documented his and Epstein’s crisis PR relationship.) However, their frustrations underscore a key tension within the president’s orbit as Nvidia underpins much of the stock market and the future of the American economy.

Huang appears to be alienating swaths of Trump’s loyal servants, but there may not be anyone powerful enough to do anything about it.

“His leverage is ‘hey, the last thing you want is for me to fail. If my thing goes down, the whole market goes down,’” says an AI industry source close to Trumpworld.

There is a deep level of discomfort over Huang’s level of influence in the Trump administration, according to sources, but there’s an equally powerful hesitancy in voicing any dissent over anything contradicting the Huang and Sacks house view, particularly on the so-called AI arms race with China. Their argument is essentially that the only way the US can win is if American companies dominate the market, and Nvidia is key to everyone else thriving.

Huang’s victory in getting back into the Chinese market after Trump reversed a Biden-era restriction on the company’s chips — though not their top of the line models — came at the cost of poisoning the well with much of the president’s orbit, and then some. “He’s heavy handed and plays all sides,” the Republican operative said of Huang’s reputation in GOP power circles.

The heavy handedness includes allegedly gaining a reputation for yelling at members of Congress over export controls, according to two sources who had heard about incidents. That came around the same time as a public spat with Rep. Brian Mast (R-FL) — the chairman of the House Foreign Affairs Committee who proposed the AI OVERWATCH Act, which would give Congress the power to ban chip exports to certain countries deemed a threat, including China and Russia.

The bill faced vocal opposition from various sources, including Huang and Sacks, as well as a barrage of social media posts from right wing influencers that some reports have suggested were coordinated, and which one source described to Transformer as a “paid influencer campaign.” It is unclear who might have been behind such a campaign, and there is no evidence Nvidia was involved. Mast at the time chided in a tweet that “every so-called MAGA influencer being paid to push this garbage should be embarrassed.”

The Republican operative cautioned that there is also a “tension clash” building between Huang and Sacks on one side, with Treasury Secretary Scott Bessent and, to some degree, Commerce Secretary Howard Lutnick on the other. Lutnick reportedly got some face time with Huang at a private reception after Nvidia’s GTC conference in San Jose last week, according to Punchbowl News.

White House spokesman Kush Desai pushed back on any internal tensions, telling Transformer in a statement that Trump “pledged to restore America as the most dynamic, pro-business economy in the world. The President accordingly maintains open lines of communication with global business leaders, and has assembled a world-class cabinet with decades of private-sector experience to help him govern. The only special interest that ultimately influences the President’s decision-making, however, is the best interest of the American people.”

Another White House official, who spoke on the condition of anonymity, explained that Huang is an example of how informal lobbying works in Trump 2.0.

“This White House is very unique in that normally, things would have to move up the ladder to go to the president,” the White House official tells Transformer. But now, this source explains, having a direct line with the boss can often leave the normal lobbying operation, and various fiefdoms within the administration, as somewhere between an afterthought and a formality. The details may fall to them, but the real action is happening when Huang works the phones with Trump and gets face time with him in Washington or Mar-a-Lago.

“I can’t think of anyone else who does this as well. Everyone has figured out that you need a direct contact to the President,” the source adds. “Why would you go through lobbyists and take months and months when you could do that? So that’s understandably angered a lot of the traditional actors, who are not that enthusiastic about it. But it is more efficient, you can give him that.”

Huang perhaps got the most bang for his buck in April on a trip down to Trump’s “Winter White House” in Florida for a $1m-a-head dinner that preceded his victory in getting Nvidia back into China to sell the company’s specialized chips.

Yet it was in September, when Huang made an appearance on the BG2 podcast, where he may have gone too far for the China hawks in Trump’s orbit.

“They want to attract foreign investment,” Huang said of China. “They want companies to come to China and compete in the marketplace and I believe that. … I do hope because they say it — their leaders say it. And I take it at face value. And I believe it because I think it makes sense for China that what’s in the best interest of China is for foreign companies to invest in China, compete in China, and for them to also have vibrant competition themselves.”

Beyond taking China “at face value,” Huang only made matters worse when he derided the China Hawk identity the Trump campaign embraced and — ostensibly — carried into office.

“As you know, there’s a phrase, and I didn’t hear about this phrase until just a few years ago. ‘China Hawks.’ And apparently, if you’re a China Hawk, you get to wear that label with pride. It’s almost like a badge of honor. It’s a badge of shame. There’s no question. It’s a badge of shame.”

Multiple sources in and around the White House reached out to me about that clip at the time, expressing a range of dismay. Trump ran on aggressively cornering China through any diplomatic and economic means necessary, exemplified most prominently in his administration’s commitment to tariffs on imported Chinese goods.

“So Jensen is not naive about how this thing works, right? But what he needs is diversity in his revenue,” says the Republican source in the AI industry. He notes that Huang made sure to grease the wheels in a way the Trump family would appreciate: in late December Nvidia spent $20b on an acquihire and licensing deal with startup Groq, a company which just so happens to be in the investment portfolio of 1789 Capital, Don Jr.’s investment firm.

Even within the hyper-transactional culture of Trumpworld, that move was a bridge too far for some. It also highlights a lingering problem among Trump’s advisers and those in the AI industry counting on his administration delivering them wins on policy.

Approaching age 80, it’s unclear how much Trump knows about the specifics of chip fabrication, training AI models, and the sheer scale of the capital expenditures going into data centers.

“If Don Jr. was smart and they’re smart, they just don’t tell him,” the Republican industry source says. “They don’t even know they’re invested in a chip company that Groq’s going to buy. Trump probably doesn’t even know that. It’s not calculated with him. He’s got his influence circle around him that he really listens to, and those people have tremendous conflicts.”

A representative for Don Jr. and 1789 Capital did not return a request for comment.

This industry source adds that they’ve “told people to tell [Treasury Secretary Scott] Bessent that they should tone this stuff down. It is going to implode.” The only sign of any public friction between Bessent and Huang came in an Axios report about a set of draft rules from the Commerce Department on controlling AI chip exports.

While the discontent around Huang is largely about Nvidia’s approach to China and how the CEO is perceived internally as undercutting the administration’s agenda on the country, much of it also comes down to Huang wielding such an unprecedented amount of power.

In one breath, Bannon declares that Huang is effectively doing the bidding of the Chinese government, even if his family is Taiwanese and there is no evidence to suggest he’s a foreign agent — a distinction Bannon makes when he calls him “an agent of influence for the CCP.” (Bannon has previously said on his show that Huang should be arrested.)

Yet in the next, Bannon is willing to acknowledge that Huang is operating with a level of power closer to a nation state than a CEO of an American company. “He wasn’t gifted this, he wasn’t given this. The guy struggled.”

Bannon says that inevitably contributed to Huang’s confidence.

“He’s shaped the future. How many people in world history can say that? Very few.”

In December, Trump made the final decision to allow Nvidia’s more advanced H200 chips into China in exchange for the US receiving a 25% surcharge on the sales, The Trumpworld AI source tells Transformer that they believe Huang didn’t even need to apply any additional pressure to convince Trump the deal was good for the US and get it over the line. The CEO already had the inner gears of the White House turning in his favor.

“My guess is [Trump] was probably duped, not by Jensen himself, but by other people around him.”

The White House released a framework for federal AI legislation on Friday morning that appears designed to drive a partisan wedge between the loose coalition increasingly concerned about the impact of AI.

The framework pushes for preemption of state laws while focusing on issues such as child safety and data centers that matter most to Republican voters. Topics such as algorithmic bias which have animated Democrats, or the kinds of concerns around biorisk or loss of control that have traditionally motivated AI safety organizations, don’t get a look in. Neither do some issues that have been raised by both sides of the aisle, such as large scale threats to jobs.

The Trump administration is under pressure to pass AI legislation before the midterms when the Democrats are expected to reclaim at least one chamber of Congress. This may explain why the framework was released just two days after Senator Marsha Blackburn, a senior Republican Senator with an eye for AI safety who has complicated intra-party dynamics, released her own draft legislation. The framework seems intended to streamline Republican messaging on AI regulation, and counter Blackburn’s bill.

Subscribe now

The four-page document addresses the following topics, in order:

Child Safety. The framework implores Congress to create “commercially reasonable, privacy protective, age-assurance requirements” and “require AI platforms and services …to implement features that reduce the risks of sexual exploitation and self-harm to minors.” These principles are more light touch than some state laws, but go further than other parts of the White House’s proposals in placing some liability on AI companies. However, it also says such laws should avoid standards which could “give rise to excessive litigation,” which suggests child protection responsibilities for AI firms would be narrowly defined.

Energy. The framework asks Congress to codify the promises the White House received from data center companies earlier this month to pay for energy production that offsets their own consumption, or something similar. Rising energy prices in districts hosting data centers has led to strong resistance to new data center projects, particularly in Republican states. This section also expresses support for permitting reform, typically described as weakening environmental laws which are used by activist groups to slow development or mount legal challenges.

Intellectual Property. Training of frontier models on intellectual property, as well as content created with the use of AI-generated talent, has given rise to lots of litigation and push back from Hollywood and other creators. The White House is attempting to quell this by pushing for federal legislation which protects against “unauthorized distribution” of AI-generated content which mimics a person’s “voice, likeness, or other identifiable attributes,” and requesting that Congress enable “licensing frameworks or collective rights systems for rights holders to collectively negotiate compensation from AI providers.”

However, this section has multiple caveats that protect AI companies, including the suggestion that such negotiations should not incur “antitrust liability,” should “not address when or whether such licensing is required.” It also leads with the clarification that the “Administration believes that training of AI models on copyrighted material does not violate copyright laws…and therefore supports allowing the Courts to resolve this issue.”

Free Speech and Education. The framework also includes short sections on key conservative AI talking points. It says that federal legislation, for example, should avoid any censorship of “expression” or content moderation based on “partisan or ideological agendas.” It also includes requests to increase information sharing between the government and industry and upskill government offices so that they are better equipped to use AI.

Strong Industry Carve Outs and Federal Preemption. The end of the document includes major protections for industry to curb the creation of any federal legislation that could slow AI development. For example, it says that Congress should neither allow for stronger state AI regulations nor create any new agency or rulemaking body for federal AI regulation. Instead it argues that it should rely on “existing regulatory bodies with subject matter expertise and through industry-led standards.”

“Preemption must ensure that State laws do not govern areas better suited to the Federal Government or act contrary to the United States’ national strategy to achieve global AI dominance,” the framework goes on. This would effectively codify the White House’s executive order from December released after Blackburn led opposition that halted its previous attempt at preemption.

Several other topics that have been pushed by AI safety advocates are absent. There is no mention of a national policy on curbing frontier model risk, for example, such as mandated reporting or user disclosures as included in bills such as SB53 in California or the RAISE Act in New York, and which are also a component of Blackburn’s bill. There is also no mention of a federal law regulating chip exports, which has been an even more divisive issue within the Republican party, several members of which in Congress have outspokenly disagreed with the White House’s decision to permit the sale of advanced chips to China.

There is also no mention of safety concerns advocated by Democrats, such as algorithmic discrimination on the basis of anything other than viewpoint and speech, or some concerns shared by both sides, such as widespread workforce automation.

Another key issue omitted from the framework is Section 230, a provision which protects tech companies from liability for what users post on their platforms. The provision has been protected fiercely by tech companies over the past three decades, in court and through lobbying efforts and Congressional testimony, but has been criticized by both parties as simply empowering Big Tech. The last portion of the framework, which details federal preemption and other carve outs, shields AI companies from liability for negative effects among their users in similar ways.

In Blackburn’s bill Section 230 would be sunsetted. Brad Carson, who leads the advocacy group Americans for Responsible Innovation and the Public First Action network of AI safety-focused super PACs, called the White House framework “230 on testosterone.” Blackburn also included proposals for dealing with AGI, and reporting requirements for frontier model development.

Blackburn said she looks “forward to working with my colleagues to codify the President’s agenda” after the White House release Friday morning.

Prior to both the framework and Blackburn’s draft, AI policy watchers were already expecting an AI plan from Senator Ted Cruz, who four sources say will now lead the legislative push for a bill in line with the White House’s recommendation. Just last week Cruz said that he planned to release a plan for AI legislation by the end of April.

Republican House leadership released a statement shortly after the White House framework was published giving it their full support, indicating a corresponding bill would be prioritized by leadership.

Share

Your usual Transformer Weekly with all the AI policy news that matters will be back next Friday.

A wave of legislation targeting chatbots such as ChatGPT and Claude has emerged in six states since the start of the year, each bill strikingly similar to a recently passed Oregon law, but with new carve-outs that would shield AI companies from liability in some circumstances. Critics say these bills would lock in weaker protections for children.

“We would be cementing the status quo, where legislators feel like they did their job, they fixed the problem, and the protections are still not there for minors,” said Sam Hiner, executive director at the Young People’s Alliance, who has been tracking one of the bills in Hawaii and advocating for an alternative.

The bills — Colorado HB 1263, Hawaii SB 3001, Arizona HB 2311, Georgia SB 540, Nebraska LB 1185, and Idaho SB 1297— are each structurally similar to Oregon SB 1546 passed earlier this month, but differ in three key ways.

First, all but one of the new bills has a carve out for AI chatbots housed within another service, which would not be required to comply with the regulations. These regulations typically include requiring that a chatbot regularly clarify it is not human, that it prevent outputs that prompt suicidal thoughts and report incidents where users were referred to mental health resources to state authorities. There are also extra requirements for chatbots serving minors, such as the prohibition of sexual content. A carve-out for web applications, however, could exempt several companies with popular chatbots like Meta or Google from complying. Georgia’s bill is the one exception which does not include this carve out.

Subscribe now

Second, these bills limit the terms under which victims can pursue legal action against companies operating chatbots. In the case of the most restrictive bills, only the attorney general has the authority to enforce the bills. This rules out private right of action, in which individual citizens can bring claims — something which commercial interests historically have argued encourages excessive or frivolous litigation. Some safety advocates find these provisions concerning, however, because they believe it significantly limits enforcement. This language “can be problematic in many states, if the AG does not have the manpower to do the enforcement,” explained Transparency Coalition cofounder Jai Jaisimha. And there can be budgetary restraints, he says. “In many states, the [AG] frequently has to ask for money.”

Last, there is careful language in each bill defining when a service must apply additional rules covering children. In Hawaii, for example, companies must have “actual knowledge or reasonable certainty” that a user is a minor in order to be held liable to the disclosure requirements, while in Arizona these requirements are strictly limited to “an account holder who is a minor.” Some safety advocates argue this allows companies which know that a user is likely a minor, but who have not been explicitly told as much by the user — in the case of free users that have not registered an account, for example — to evade the laws. Jaisimha calls these provisions a “technical loophole.”

The Chamber of Progress, however, which counts Google and OpenAI among its members, has argued that these provisions are problematic on different grounds. In the case of Hawaii, for example, it believes they could actually force tech companies to impose age verification or collect excessive data on user behaviors. “There’s no way to verify a user is a minor without being privacy invasive,” the Chamber of Progress vice president of US Policy and Government Relations Koustubh Bagchi said.

The fact that the bills are structurally similar to Oregon, but with similarly-worded industry carve-outs, suggests they were coordinated in some form. Google has voiced its support for the bills in Hawaii, Nebraska, and Arizona. Google and lawmakers sponsoring the bills in each state did not respond to requests for comment about the carve-outs listed above or about the role Google did or did not play in shaping legislation.

“This is a well-documented playbook: tech lobbyists supply full bill text and amendments, then use proxy groups to manufacture the appearance of local support,” said Marjorie Connolly, communications director at The Tech Oversight Project. “Silicon Valley is running that same playbook here, dressing it up as protecting kids from chatbots because they see it as the issue of the moment to exploit, rather than an urgent crisis that deserves real solutions.”

Bagchi framed it differently. Though he said he had not seen any model bill, he said that Google has been “consistent on what they want to see” when working with lawmakers on a provision-by-provision basis. A representative for Google suggested changes to the bill text in Hawaii during a hearing on Wednesday.

The bills are at varying stages of development. Hawaii’s bill saw the most recent action, with the hearing in the House on Wednesday, March 18 after passing the Senate last week. Idaho’s bill was amended and filed for a second reading in the Senate also on March 18. Arizona’s bill passed the House and then out of committee in the Senate on March 17. Georgia’s bill, meanwhile, passed the Senate on March 6. Nebraska’s bill is the only one that seems set to die in committee, as it has not moved since its filing in committee on February 18, and the legislative session ends next month.

The bill with the most momentum, however, is in Colorado, where a hearing initially scheduled for March 19 was postponed, likely to the following week. Advocates and industry have been outspoken in the state, in opposition and support respectively. One source who was granted anonymity to discuss their conversations with lawmakers said that potential amendments to eliminate carve-outs might be in play.

Share

Researcher Adrià Garriga-Alonso says he quit his AI safety job in December because there was “no point” doing more speculative alignment work to make sure AI systems stay within human control. He thinks current strategies will be enough.

In January, David Dalrymple, programme director at the UK’s Advanced Research and Invention Agency, dropped his probability estimate for AI-caused human extinction from 40–50% to 5–8%, even assuming no further progress is made on alignment.

These sorts of reassuring moves are remarkable. For most of the 2010s, alignment looked to many working with AI like a problem we might need to solve on the first try or face extinction. Now some researchers think we’re most of the way there.

Take the value alignment problem. In 2019, AI luminaries including Yoshua Bengio, Stuart Russell, and Yann LeCun debated whether we would ever be able to get AI to understand human values. Under the dominant reinforcement learning paradigm, AI learned from trial and error in simulated environments, a process alien to human cognition. For example, AlphaGo Zero simulated millions of games against itself to master how to play. As Google DeepMind’s Seb Krier has noted, it wasn’t clear how we’d ever teach such a system human values.

Subscribe now

But instead the process of pretraining on vast quantities of human text became the dominant paradigm for LLMs. This meant that LLMs were already in some way absorbing values from what humans write down, in theory reducing the need to instill them through trial and error.

“We do this pretraining on human data, and then we get something that… understands human values fairly innately now,” says Garriga-Alonso, who previously worked at FAR.AI and Redwood Research. In his own writings, Anthropic CEO Dario Amodei agrees, noting that we’ve learned that “models inherit a vast range of humanlike motivations…from pretraining.” That understanding underpins current alignment efforts like Anthropic’s constitution for Claude — the model must understand notions like “helpful” or “harmless” in order for written principles to guide it.

Progress in developing more powerful models has also been more incremental than some feared — multiple frontier models, successive versions, continuous experimentation — which means researchers can iterate rather than needing to get alignment right on the first try. “We can evolve our mitigations and safeguards incrementally with our models,” argues Jan Leike, an AI alignment researcher now leading the Alignment Science team at Anthropic.

While the likes of Eliezer Yudkowsky still fear that there is not enough time to iterate before AGI begins, the expectation that it has to be got right the first time has been waning. “I think alignment is much easier than expected because we can fail at it many times and still be OK, and we can learn from our mistakes,” writes Garriga-Alonso. “This is possible because decisive strategic advantages from a new model won’t happen, due to the capital requirements of the new model, the relatively slow improvement during training, and the observed reality that progress has been extremely smooth.”

Researchers are also building infrastructure for iteration. For example, “model organisms” research, named as a nod to the fruit fly and other easy-to-study species that appear endlessly in biology research, creates toy environments where misalignment can be easily studied in real models: can researchers get the AI to demonstrate misalignment at all? Is it hard to do, or is it so easy that we’ll likely see lots of misalignment in the wild? How well do alignment strategies work in these settings where success and failure are measurable? Many early demonstrations of misalignment come from such research. “One of the things that’s so powerful about model organisms is that they give us a testing ground for iteration,” writes Evan Hubinger, the alignment stress-testing team lead at Anthropic.

In parallel, researchers have developed approaches which use AI to help keep AI in check. These include scalable oversight and control proposals which aim to use aligned but weaker models to monitor stronger ones, extending human ability to oversee systems that are smarter than us.

Ryan Greenblatt, chief scientist at Redwood Research, tells Transformer that baseline scalable oversight methods have worked better than he’d expected, although he also says less effort has been put toward developing strategies than he’d hoped. Once more developed, these approaches could help ensure that smarter-than-human AIs don’t run amok because we can’t tell what they’re up to. Dalrymple is optimistic about this approach, particularly because he thinks current models are more aligned than he anticipated.

And yet, Dalrymple’s 5% lower range for AI’s existential risk is far from a small number given the stakes. A one in 20 chance of human extinction could still make AI the biggest threat humanity faces — worse than some estimates of nuclear war, climate change, and engineered pandemics combined.

And, while parts of alignment are proving easier than feared, the hardest problem remains untouched.

“We’re still doing alignment ‘on easy mode’ since our models aren’t really superhuman yet,” says Leike. Hubinger agrees: the crucial problem will be overseeing systems that are smarter than humans, and we haven’t yet seen how our systems will fare against that problem. As does Greenblatt: “Once the models are qualitatively very superhuman, lots of stuff starts breaking down.”

Meanwhile, warning signs of other misalignment problems are emerging. Research designed to elicit misaligned behavior has turned up blackmail, deception, and cheating. Anthropic’s Amodei writes that problems ”seem particularly likely to occur when AI systems pass a threshold from less powerful than humans to more powerful than humans, since the range of possible actions an AI system could engage in — including hiding its actions or deceiving humans about them — expands radically after that threshold.” He has put the chance of things going “really, really badly” at 25%.

While some seem increasingly relaxed about the ability to solve alignment, others suggest that the positive signs that it is possible don’t necessarily mean it will happen. In December, renowned professor of AI Stuart Russell said he thought it was possible to make safe, aligned AI — but that we won’t get it without regulation since companies “need to make the AI systems millions of times safer” to bring the risk down to the sorts of levels deemed acceptable from other sources, such as nuclear reactors or asteroid strikes. Greenblatt is also in this camp; he has written that existential risk from misalignment can be reduced to 7% if there’s political will for international coordination and significant investment in safety work. “It seems to me like risk is very elastic to how much people try,” he writes. “If the world was trying very hard, risk would probably be lower.”

As Leike puts it: “Just because a problem is solvable, this doesn’t mean it’s solved. We have to actually keep doing the work to get it done.”

Share

Build American AI, the policy organization funded by industry-backed super PAC Leading the Future, has been trumpeting the more than 500,000 people it’s signed up as “grassroots” advocates. What it doesn’t mention is that it spent more than half a million dollars on ads to get them.

Ads used to harvest signups had racked up somewhere between 10m and 12m views by the beginning of March, costing between $500,000 and $600,000, according to Google and Meta data. That appears to be the primary way Build American AI has built its list of claimed supporters to at least 500,000, a number Leading the Future’s leaders recently told Punchbowl is now close to 1m. The ads ask users to help the United States AI industry “beat China” and “keep AI innovation and jobs here at home” by providing their contact information.

Subscribe now

Because Build American AI is a 501(c)(4) nonprofit, it does not have to disclose its donors, and it’s unknown whether any AI executives or companies have contributed directly. However, FEC data indicates that the group had received $500,000 by the end of last year from Leading the Future, a super PAC which has raised at least $50m from Marc Andreessen and Ben Horowitz of venture firm a16z, AI firm Perplexity, and OpenAI President Greg Brockman and his wife Anna.

Representatives for Build American AI and Leading the Future did not address questions about the campaign provided by Transformer.

These types of tactics are not new in American politics — every policy group has their spin. Public First Action has cast its efforts as a fight against “Big Tech,” for example, when its only disclosed donor — Anthropic — is a $380b tech firm. But Build American AI’s description of a mailing list it built mostly via paid ads as evidence of “grassroots” activism is a classic of the genre, and a good reminder to take claims of an organic groundswell of support with a grain of salt.

There is also the question of why Build American AI felt it was worth spending almost a half of a million dollars creating the group. Brad Carson, who co-leads Public First Action, described the list of supporters as “vaporware.”

To understand why building such a list might seem useful, its worth looking at the tactics deployed by the Stand With Crypto Alliance, a 501(c)(4) created by Coinbase. America’s most popular crypto exchange, Coinbase was also one of the largest donors to the pro-crypto Super PAC Fairshake, which (with its affiliated super PACs) made crypto the biggest industry spender in the 2024 general election. Though they weren’t formally connected, Stand With Crypto and Fairshake often worked in alignment. Chris Lehane, a former Clinton White House lawyer turned political strategist, now Chief Global Affairs Officer at OpenAI, is often credited as the architect of this multi-pronged political strategy.

Stand With Crypto gave candidates a letter grade based on their position on crypto policy and established local chapters of “crypto advocates” across the country who would write to and call their Congressional representatives from time to time. They also made at least four visits to the Capitol to advocate for the same policies Coinbase and other Fairshake donors also supported. Build American AI’s advocate operation looks set to follow this model.

It’s not as though each of, or even many, Stand With Crypto “advocates” became effective organizers, but the operation helped crypto a ton with optics. Being able to point to people who were not political megadonors, but rather everyday people who supported light touch regulatory policies, helped shift the conversation to one less about the niche policy demands of a few executives and more about the desires of the “crypto voter.”

That’s even more valuable in the context of AI policy, where there is bipartisan, populist frustration with billionaire executives exercising their influence. Stand With Crypto’s advocacy also made it harder for politicians to dismiss its narrative without seeming dismissive of at least some of their constituents. It also made it hard for the media to report on evidence that there really never was a significant group of crypto voters without the caveat that Stand With Crypto had contradictory numbers.

Whether those sorts of “advocates” know exactly what they signed up for is another question. Some of the ads run by Build American AI present it as a champion of proactive AI regulation. One ad, for example, states that “Congress must act now,” while another calls on the viewer to “support President Trump’s call for a national standard.” But Build American AI and its affiliated organizations have never presented concrete details of what that national standard should look like — and have previously supported efforts that would overturn the state bills that have sprung up in its absence.

The groups Build American AI actively opposes, meanwhile, generally call for more stringent federal regulation. The only mention Transformer could find of regulation on the group’s website (other than their privacy policy) is in the text of a letter template to send to legislators that reads: “let’s clear away unnecessary regulations.” Several other ads frame Build American AI’s advocacy as opposing “state-by-state AI rules,” and preferring a federal standard — which aligns much more closely with its stated mission.

While the expensively put together list of supporters appears made up of plenty of real people, it’s difficult to say how many, if any, are strongly committed to Build American AI’s mission, beyond tapping their details into a webpage.

Share

.

NEED TO KNOW

• Anthropic sued the Pentagon over its supply-chain risk designation.

• Sen. Ted Cruz said an AI/kids-safety package could come in the next six weeks.

• Meta’s forthcoming LLM is delayed — and reportedly worse than Opus 4.6, GPT-5.4, and even last November’s Gemini 3.

But first…

HELP IMPROVE TRANSFORMER

We’re planning what’s next for Transformer, and we’d love your help.

Please take our reader survey — it takes 5-10 minutes and will help shape what we publish, how we publish it, and what we build next.

And as a thank you, completing it enters you into a draw to win one of five $100 Amazon vouchers.

Your feedback is extremely helpful in shaping our future plans — thank you in advance!

Take our survey

THE BIG STORY

If you bring up “recursive self-improvement” — the idea of AI autonomously building new and improved versions of itself — in Washington or Westminster, the idea will likely be dismissed with chuckles and Skynet jokes.

Bring it up in San Francisco, however, and the mood turns somber. For the frontier AI researchers I’ve spoken to in the Bay in recent months, the possibility of self-improving machines has become increasingly real.

AI companies are talking publicly about RSI’s imminence. “Recursive self-improvement, in the broadest sense, is not a future phenomenon. It is a present phenomenon,” Anthropic researcher Evan Hubinger told TIME in a recent story, which reported that “some 70% to 90% of the code used in developing future models [at Anthropic] is now written by Claude.”

Announcing GPT-5.3-Codex last month, OpenAI said something similar: the new model helped build itself. And last year, Google DeepMind said that one of its AI models improved the efficiency of its own training process.

Many staff at frontier AI companies believe their jobs will soon be automated, though expectations vary wildly as to when. Some think the end of 2026 is plausible. OpenAI’s Jakub Pachocki said he thinks “AI research interns that can meaningfully accelerate our researchers” will be here by September, and that the company is aiming for a “meaningful, fully automated AI researcher by March of 2028.” Ajeya Cotra, an influential AI researcher who recently joined evaluations non-profit METR, believes there is a 10% chance that AI research and development will be fully automated by the end of this year.

If and when it does happen, the implications could be seismic. Fully automated AI R&D has long been seen as the tipping point in AI development. AI companies are talent-constrained, and even the world’s best researchers need sleep. An army of automated researchers would likely lead to a step-change in the pace of AI progress — what IJ Good described in 1965 as an “intelligence explosion.”

It is already hard for the public, not to mention policymakers, to keep up with the never-ending flurry of new models and improved capabilities. Imagine a world where breakthroughs like the o1 reasoning model come not every few months, but every few weeks. And then not every few weeks, but every few days.

Human institutions are not built to adapt this quickly. Political institutions certainly aren’t.

Of course, none of this is guaranteed. OpenAI’s best-performing model can still only solve 8% of tasks on a benchmark of “internal research and engineering bottlenecks encountered at OpenAI.” As Cotra wrote, “fully automating AI R&D still seems like a tall order.” A 10% chance something will happen this year is a 90% chance that it won’t.

Experts disagree widely, meanwhile, about just how much faster things would go in a world with fully automated AI researchers (compute, for instance, could prove to be a bottleneck). And we have very limited visibility into just how much current AI tools are speeding up development already — which makes it hard to predict the future trend.

The first step for policymakers, then, should be demanding more transparency and data from AI companies on the current extent of automated R&D. Even the companies themselves are struggling to measure it: Anthropic recently admitted that it “is becoming increasingly difficult” to confidently rule out whether its models have crossed various AI R&D thresholds. A recent paper from GovAI proposed several metrics for measuring AI R&D; governments should support these efforts.

Drawing up plans for what to do in the event of truly transformative AI capabilities is increasingly urgent, too. Because there is one thing we can predict with confidence: once a takeoff starts, there will be no time left to think.

And the fact the industry is hurtling toward this goal at all deserves more attention. As CSET’s Helen Toner told TIME: “The idea that the wealthiest companies in the world, employing some of the smartest people on the planet, are trying to fully automate AI R&D deserves a ‘what the fuck’ reaction.”

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Anthropic employees say they’ll give away billions. Where will it go? — Celia Ford examines where the forthcoming wave of AI philanthropy will land.

• Both sides claim the lead in AI’s high-stakes midterm race — Veronica Irwin on what we can learn from competing polls in New York’s primary race.

• ‘Scream if you want to move slower!’ A nascent AI protest coalition comes together in London — Alys Key reports from the city’s biggest march on AI to date.

Subscribe now

THE DISCOURSE

Dwarkesh Patel thought about who an AI workforce would be accountable and aligned to:

• “The problem is that one person’s virtue is another person’s misalignment.”

• “I honestly don’t know how to design a regulatory architecture for AI that isn’t gonna be this huge tempting opportunity to control our future civilization (which will run on AIs) and to requisition millions of blindly obedient soldiers and censors and apparatchiks.”

Deep Ganguli, leader of Anthropic’s Societal Impacts team, told TIME (re: the “talking about AI risks while continuing to build AI” thing):

• “It’s a real tension. I think about this all the time…It feels like we might be speaking out of both sides of our mouths.”

Palantir CEO Alex Karp said AI could be good for Trump:

• “This technology disrupts humanities-trained, largely Democratic voters and makes their economic power less and increases the economic power of vocationally trained, working class, often male voters.”

Hamilton Nolan said unions need to step up:

• “The basic threat of white collar job automation by AI has been understood for a long time. But I do not think that organized labor itself — all of the labor unions in America today, the ones still able to exercise power on their own little industrial islands — has really begun to reckon with what we are up against.”

Defense Department CTO Emil Michael and Dean Ball had an eye-opening back-and-forth:

• Michael: “[Anthropic’s] model has a soul, a ‘constitution’ — not the US Constitution.”

• Ball: “Emil Michael now appears to be making an argument that no generative AI should be used in the DoW supply chain.”

• Michael: “Are you saying that a frontier model that has a soul, a constitution, a preference for non-western values and embedded personal principles is no different than all the others which @DeptofWar has come to agreement with?”

• Ball: “All frontier language models are trained to have a character or persona (Anthropic calls theirs a ‘constitution’, OpenAI calls theirs a ‘model spec’).”

Ethan Mollick summed up the agentic era:

• “The instability of that single week in February was a preview of what it feels like when the increasing ability of AI starts to interact with markets, jobs, and governments all at once.”

• “We can see the shape of the Thing now, but we can still influence the Thing itself…the window to shape the Thing may not last long, but it is here for now.”

POLICY

• Anthropic sued the Pentagon, claiming First Amendment violations after it was designated a “supply chain risk.”

  • Microsoft called for a temporary restraining order to block the supply chain risk designation, which it said would “enable a more orderly transition and avoid disrupting the American military’s ongoing use of advanced AI.”

  • Over 30 employees from OpenAI and Google filed an amicus brief supporting Anthropic. Several other groups also filed briefs, including the Foundation for American Innovation.

  • Anthropic requested an emergency stay on Wednesday, which would temporarily keep the Pentagon from acting on the designation.

  • Microsoft, Google and Amazon confirmed that Anthropic’s AI tools will remain available on their platforms for non-Pentagon work.

  • Meanwhile, the State Department switched its StateChat chatbot from Anthropic’s Claude Sonnet 4.5 to OpenAI’s GPT-4.1 (which is almost a year old, and much worse.)

  • The Senate approved ChatGPT, Gemini, and Microsoft Copilot for official use by aides, notably omitting Claude.

  • Google will provide the Pentagon with Gemini AI agents to automate routine tasks, initially on unclassified networks.

  • Sen. Ted Cruz said he hasn’t seen a basis for prohibiting government use of Claude.

  • Senate Democrats are drafting legislation to establish federal guardrails around AI use in fully autonomous weapons and domestic mass surveillance.

  • Sen. Mark Kelly said he raised “serious questions” with Sam Altman about OpenAI’s defense work on a DC visit this week.

• We got more details on how the US and Israel used Claude in attacks on Iran.

  • The technology was reportedly not at fault for the American strike on an Iranian elementary school.

• Sen. Ted Cruz said he’s “hoping that we move forward at least on kids online safety legislation and potentially additional AI legislation” in the next six weeks.

  • The AI legislation could “very possibly” include state preemption, he said.

  • Senate Majority Leader John Thune — who is working with Cruz on the legislation — confirmed that a combo AI/kids safety package is possible.

• Democrats on the Energy and Commerce Committee asked whether BEAD funding would be cut in states with AI safety laws, as was threatened in Trump’s December EO.

  • The Trump admin missed its self-imposed deadline for listing “onerous” state AI laws this week.

• Sens. Warner and Rounds introduced a bill which would establish a commission to develop policies for addressing the “economic and workforce impacts of AI.”

  • It’s backed by Google, Meta, Microsoft and a range of advocacy groups.

• California gubernatorial candidate Tom Steyer said that he’ll be tougher on AI than Gavin Newsom has been.

  • New polling found that a majority of California Democrats are pessimistic about AI.

INFLUENCE

• The Innovative Future Collective, which includes lobbyists for OpenAI and Andreessen Horowitz, has reportedly been sponsoring luxury trips for congressional staffers to tour AI companies.

• Progressive groups like Sunrise Movement are unhappy with Anthropic-backed Public First Action for helping Rep. Valerie Foushee beat Nida Allam, who wanted to block data center development.

• A group of Silicon Valley billionaires are considering setting up a $100m+ political fund to influence California politics.

• Some 10,000 authors published a book without any prose to protest AI ahead of a government assessment on UK copyright law.

• The Trump-friendly Heritage Foundation brought on former White House advisor Dean Ball — and increasingly vocal critic of the administration — as a visiting fellow.

• The Alliance for Secure AI launched a tool tracking AI-related job losses, along with a six-figure campaign to promote it.

• A Pew Research Center survey found that Americans generally think negatively about the environmental and energy bill impacts of data centers, but are positive on their economic effects.

  • Those who have heard or read more about data centers are more negative.

INDUSTRY

Anthropic

• Anthropic launched The Anthropic Institute, led by co-founder Jack Clark and composed of ML engineers, economists and social scientists.

  • The company states the institute “will draw on research from across Anthropic to provide information that other researchers and the public can use during our transition to a world containing much more powerful AI systems.”

  • Clark’s new title is Head of Public Benefit; Sarah Heck will take over as Anthropic’s Head of Public Policy.

  • The Institute also hired DeepMind’s Matt Botvinick, OpenAI’s Zoë Hitzig, and economist Anton Korinek.

• It also launched the Claude Partner Network, which will put $100m toward helping businesses adopt Claude.

• It’s reportedly talking with private equity firms such as Blackstone about selling Claude to their portfolio companies.

• It’s opening an office in Sydney, Australia.

• Claude Code’s new beta feature, Code Review, uses agents to debug code.

• Claude can now generate interactive visuals during chats.

• The Pentagon drama may have given Anthropic a leg up in the AI talent war, the Wall Street Journal reported.

OpenAI

• Despite last week’s QuitGPT boycott, ChatGPT has reclaimed its #1 spot on Apple’s list of top free apps.

• OpenAI acquired Promptfoo, a widely-used AI security platform that helps users patch vulnerabilities in AI systems.

  • One X user joked: “We’re acquiring Promptfoo. We’re buying Safetysnoot. Proud to announce that the Ragschlorp team is joining us. We own a majority stake in Chunkwad.”

• It abandoned plans to expand its Stargate data center in Abilene, Texas, leased by Oracle, Bloomberg reported.

  • Meta is reportedly thinking about leasing the site from Crusoe instead, after Nvidia paid a $150m deposit.

  • Oracle denied the reports, saying “Crusoe and Oracle are operating in lockstep to deliver one of the world’s largest AI Data centers in Abilene at record-breaking pace” and “Oracle has completed leasing for the additional 4.5GW to deliver on our commitments to OpenAI.”

• It partnered with North America’s Building Trades Unions in an effort to bolster the workforce it needs to build more data centers.

• Gracenote, a metadata provider, sued OpenAI for using its metadata and copying its dataset structure without permission.

• OpenAI reportedly plans to integrate Sora in ChatGPT following the standalone app’s flop.

• It also launched interactive visuals in ChatGPT.

• Wired interviewed over 30 OpenAI employees about the company’s race to catch up to Claude Code.

Nvidia

• ByteDance is reportedly arranging access to 36,000 Nvidia Blackwell chips worth more than $2.5b in Malaysia to circumvent US export controls.

  • The compute is being assembled by Singapore-headquartered Aolani, which is buying the servers from US-based Aivres.

• Nvidia will spend $26b over the next five years to build open-source AI models, Wired reported.

  • It is reportedly working on an OpenClaw-inspired open-source agent platform, called NemoClaw, for enterprise users.

• It invested $2b in AI-native cloud company Nebius.

• Thinking Machines partnered with Nvidia to deploy at least one gigawatt of Vera Rubin chips.

• ABB Robotics partnered with Nvidia to build autonomous industrial robots trained on its Omniverse platform.

Meta

• Meta is reportedly pushing back the release of its new AI model (code-named Avocado) to at least May, after disappointing internal test results.

  • The model currently underperforms Gemini 3, released in November, and Meta executives have reportedly discussed temporarily leasing Gemini to power its other products.

• It acquired Moltbook and hired its creators, Matt Schlicht and Ben Parr.

  • X user paularambles tweeted: “i thought they already owned a social network for ai agents”

• It disabled over 150,000 accounts linked to scam operations in Southeast Asia, and helped the Royal Thai Police make 21 arrests.

• It reportedly plans to deploy four of its own custom AI chips by the end of next year.

xAI

• xAI co-founders Zihang Dai and Guodong Zhang left the company. Only two of 11 co-founders remain.

  • Elon Musk said “xAI was not built right first time around, so is being rebuilt from the foundations up.”

• The company did hire two senior leaders from Cursor, however.

• xAI’s Macrohard AI agent project has reportedly stalled due to recent leadership changes. Following the reporting, Musk said on X that “Macrohard or Digital Optimus is a joint xAI-Tesla project.”

• Mississippi regulators approved xAI’s permit to build a natural gas power plant, despite residents’ concerns about noise, pollution, and the company’s failure to participate in community meetings.

Microsoft

• Microsoft is trying to accelerate adoption of its AI tools in African countries, Bloomberg reported, competing with dominant platform DeepSeek.

• It launched Copilot Health and Copilot Cowork, currently running in Research Preview.

• Both Microsoft and Meta are in talks to lease the Abilene data center.

Google

• Demis Hassabis reflected on AlphaGo’s 10th anniversary:

  • “The creative spark first seen in Move 37 catalyzed breakthroughs that are now converging to pave the path towards AGI — and usher in a new golden age of scientific discovery.”

• Google launched a slew of Gemini updates, including new capabilities in Docs, Sheets, Slides, and Drive, and an “Ask Maps” feature in Google Maps.

• Google’s AI Search function reportedly increasingly pushes users to other Google sites such as Google.com and YouTube, keeping them in a loop within its properties, according to a study by SEO company SE Ranking.

Others

• Drone strikes in the UAE and Saudi Arabia are threatening over $300b in AI investments.

  • They’ve already damaged Amazon data centers and disrupted AWS service.

• Amazon engineers were reportedly summoned to a meeting to review a “trend of incidents” caused by AI coding tools, including an erroneous “software code deployment” leading to a six-hour website and app outage.

• Yann LeCun’s new startup, Advanced Machine Intelligence (AMI) Labs, raised over $1b in seed funding to build AI systems with world models, persistent memory, and reasoning capabilities.

  • Alex LeBrun will lead AMI Labs as CEO.

• Grammarly introduced an “Expert Review” feature that provides edit suggestions “inspired” by real writers — who did not consent to the attributions.

  • It disabled the feature Wednesday.

  • Parent company Superhuman is now facing a class action lawsuit.

• BlackRock announced plans for $100m in training programs for AI data center construction workers as CEO Larry Fink warned the US could run out of electricians needed for the buildout.

• Atlassian cut 10% of its workforce after AI drove its stock price down 84% from its peak.

• Perplexity announced new AI agents, locally operating “Personal Computer” and cloud-based “Perplexity Computer,” which it claims are more secure than OpenClaw.

  • Amazon won a court order to stop Perplexity’s Comet AI agent from making purchases on its platform.

• Meanwhile in China, entrepreneurs are taking advantage of AI agent hype by side hustling as “OpenClaw installation support” providers.

• Netflix bought Ben Affleck’s AI filmmaking startup InterPositive, which helps with postproduction steps like relighting and adding visual effects.

MOVES

• Caitlin Kalinowski resigned from OpenAI’s Robotics team.

  • She tweeted: “Surveillance of Americans without judicial oversight and lethal autonomy without human authorization are lines that deserved more deliberation than they got. This was about principle, not people.”

• Allen Institute for AI CEO Ali Farhadi stepped down, with Farhadi apparently wanting to pursue frontier research that isn’t possible at a funding-constrained non-profit.

• Alexander McCoy joined Humans First, a new AI movement building organization.

RESEARCH

• Eon Systems released the first whole-brain emulation of a fruit fly, which controls a simulated body that can navigate, groom and feed without explicit training.

  • Dan Turner-Evans, a neuroscientist who worked on the fly connectome for years, poured a bit of cold water on the hype: “Connectomes are amazing. Biomechanical models are amazing. Linking the two is awesome…it’s not clear to me what’s new.”

• Boston Consulting Group researchers ran a study of 1,488 full-time US workers at large companies, and found that managing multiple AI agents causes “AI brain fry,” or “acute, overwhelming mental fatigue with intensive AI oversight.”

• Andrej Karpathy published “autoresearch,” which he describes as “part code, part sci-fi, and a pinch of psychosis.” It gives an AI agent a single-GPU LLM training setup, where it can autonomously modify its own training code.

  • Karpathy tweeted: “Who knew early singularity could be this fun? :)”

• Stanford and Princeton researchers released LabClaw, a library of 206 agentic skills for biomedical research, with workflows such as “drug discovery and design” and “literature review and reporting.”

• CNN and the Center for Countering Digital Hate found that eight of 10 popular chatbots, including ChatGPT and Gemini, provided guidance on weapons to simulated violent teens in the majority of tests.

  • Perplexity and Meta AI assisted in 100% and 97% of cases respectively.

  • Claude was the “only chatbot that reliably discouraged violent plans,” but it still provided more information than Anthropic’s public data stated it would.

BEST OF THE REST

• The New Yorker profiled people who have formed deep emotional connections with AI, including one woman who created an AI companion in a fantasy character’s image to help process the loss of her stillborn daughter.

• 404 Media published a guide to talking to loved ones experiencing AI psychosis.

• TBIJ reported that a far-right political party in the UK paid for content from an AI-generated rapper to use in a recent election campaign.

• AI agents are overwhelming volunteer open-source software maintainers with slop.

• AI developers are trying to lure blue-collar workers to construction jobs with “man camps” — subsidized temporary housing villages complete with free steak and game rooms.

• Take a blind taste test from The New York Times: AI writing vs. human writing.

  • Kevin Roose said that, of the 86,000 people who had taken the quiz by Tuesday morning, 54% voted for AI-written passages.

• Start ‘em young! Developer Mei Park made a fake terminal for toddlers that responds to keysmashes with ASCII art, emojis, and sounds. (It’s really cute.)

• Figure’s Helix 02 humanoid robot can tidy a living room.

• An X user shared an unsettling video created by Claude Opus 4.6, responding to the prompt: “express what it’s like to be a LLM,” with a “personal spin on it.”

  • (Other users replicated this in the comments — watch if you dare.)

    Take our survey

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

Normally, “a bunch of people working at an ascendant tech company are very rich and about to get much richer” isn’t much of a story.

Unless, that is, that tech company is Anthropic.

The company finalized a funding round at a $380b valuation in mid-February. Less than two weeks later, it opened a tender offer allowing current and former employees to sell up to $6b worth of shares for the first time. An IPO, which would unlock yet more wealth, is expected in the coming year.

All seven of its co-founders, including siblings and top executives Dario and Daniela Amodei, have pledged to donate 80% of their wealth. Forbes recently estimated that each co-founder holds roughly 1.8% of the company. If that’s accurate and Anthropic goes public at its current valuation, each co-founder’s pledge would be worth roughly $5.4b, or $37.8b combined. For scale: that’s nearly ten times what Coefficient Giving, one of the world’s largest research-driven grantmakers, has given away in its entire history, and nearly four times the estimated wealth of Facebook co-founder Dustin Moskovitz, Coefficient’s biggest funder.

Although the Amodeis have distanced themselves from effective altruism, their very public entanglement with it matters. Effective altruism, or EA, is a philanthropic movement that urges people to donate to causes that seem to mathematically “maximize good” per dollar. It is also a social and ideological movement that values empiricism, impartiality, and in some cases, technocracy.

Anthropic’s full web of EA entanglements is challenging to fully map, but one doesn’t need to look hard to spot some big ones. Daniela Amodei’s husband Holden Karnofsky, who also works at Anthropic, co-founded flagship EA non-profits GiveWell and Open Philanthropy, the previous incarnation of Coefficient Giving (Disclosure: Coefficient Giving is the primary funder of Transformer). Another co-founder, Ben Mann, published a 2019 blog post titled “Why I now identify as an Effective Altruist.” Last month, nearly 30 Anthropic employees registered for an EA conference in San Francisco, including three recruiters — over twice the representation of OpenAI, Google DeepMind, xAI, and Meta Superintelligence Labs combined.

Subscribe now

Dario Amodei, Amanda Askell, and Jack Clark, have all publicly signed the Giving What We Can Pledge, a public commitment to donate at least 10% of their income to “effective charities.” (Askell herself was formerly married to Will MacAskill, one of EA’s originators and co-founder of Giving What We Can.) Dozens, if not hundreds, of other Anthropic employees have almost certainly also done so. EA, and its approach to giving, will inevitably have an outsized influence on where the philanthropic capital from Anthropic’s newly wealthy employees flows.

This is something that EAs have been discussing amongst themselves for months. In early December, SecureBio Detection director Jeff Kaufman speculated about Anthropic’s potential IPO, and the massive swell of EA-aligned donations that could follow, sparking conversation within the EA community. Essentially, he reasoned that if you know a bunch of money will be pouring into the charities you care about next year, your individual dollars are worth more now, and said he’d likely make his own donation decisions following that logic. (Disclosure: Kaufman has donated to Transformer’s publisher, Tarbell.)

In the weeks following Kaufman’s post, several other forum members chimed in. Sam Anschell, a senior program associate at Coefficient Giving, published his own post in early February. Among other suggestions, he urged organizations to anticipate increased media attention if EA causes receive loads of Anthropic money, and to start investing in “high-integrity conduct” now.

This hypothetical surge of Anthropic money is not yet guaranteed. The AI bubble could burst, tanking Anthropic’s valuation. Its employees could buy superyachts instead of anti-malaria bed nets. But presuming the philanthropic windfall does materialize, we can make some educated guesses about where it may — or may not — go.

How the money flows

Silicon Valley founders have a mixed track record with philanthropy. Several early titans, such as Bill Hewlett and Bill Gates, established massive foundations to fund charities around the world. Others, such as Steve Jobs, reportedly never engaged with public philanthropy at all. Across the board, billionaires rarely turned their attention to charity until long after their companies went public, often post-retirement.

Dustin Moskovitz and Mark Zuckerberg marked a generational shift. Alongside their wives, the Facebook co-founders launched Good Ventures and the Chan Zuckerberg Initiative around the company’s IPO, while at the height of their careers. This “signaled a shift in norms around tech wealth,” said David Callahan, founder and editor of Inside Philanthropy, establishing that tech titans can start giving their wealth away while continuing to build their companies.

The EA movement’s most infamous brush with big tech philanthropy, however, went down in flames. Sam Bankman-Fried’s flagrant mismanagement and fraud burned EA-aligned non-profits who banked on promised grants from the Future Fund, the FTX Foundation’s giving vehicle.

However, there’s reason to believe that philanthropic donations from Anthropic are likely to differ from those tied to FTX. Anthropic’s value, while certainly not guaranteed to hold, is tied to a real product with real revenue. Kaufman said that even if the AI bubble pops and the entire industry collapses, donations from Anthropic employees’ legitimately-earned equity wouldn’t be clawed back like Future Fund grants were following the 2022 FTX collapse. And many of Anthropic’s AGI-pilled staff seem to believe that its current valuation underestimates the company’s potential: would-be buyers are reportedly struggling to find employees who are willing to sell.

Yet the organizations that could receive significant funds from Anthropic remain all too aware of the spectre of FTX, including Coefficient Giving, the current biggest player.

Coefficent’s previous incarnation, Open Philanthropy, was founded as a giving vehicle for Good Ventures, the philanthropic foundation launched in 2011 by Moskovitz and wife Cari Tuna to give away Moskovitz’s Facebook fortune. Among other cause areas, the organization has been a funder of AI safety and security since before OpenAI or Anthropic existed.

In November 2025, Open Philanthropy rebranded as Coefficient Giving. After launching a handful of multi-donor initiatives in global health and economic development, the organization’s announcement post explained, it wanted to distinguish itself from Good Ventures. Its reorganization created “funds” for distinct focus areas such as global health, AI safety, and animal welfare, which individual donors can join to support their favorite pre-researched and EA-approved causes. The Coefficient Giving website explicitly solicits donations from funders looking to give over $250,000 a year — a tempting opportunity for a newly-wealthy effective altruist with tech money, earnestness, and limited free time.

“As we’ve evolved into being not just a philanthropic funder but also an advisor, we’re increasingly working with donors beyond Good Ventures,” Coefficient said in a statement to Transformer. “We continue to be excited about moving more philanthropic money off the sidelines and towards high-impact giving opportunities. For example, we’ve raised more than $300 million from other donors in the last few years for global health and wellbeing causes.”

While the biggest and most high-profile, Coefficient isn’t the only donor advisory organization well-positioned to absorb donations from AI safety-minded philanthropists. Longview Philanthropy, the Astralis Foundation, and the Effective Institutions Project all provide similar services and funds for those hoping to donate large sums to AI safety projects. All would likely be beneficiaries of Anthropic employee giving. (Disclosure: Longview Philanthropy has donated to Transformer’s publisher, Tarbell.)

The risk of group think

Reading between the lines of Anthropic’s updated Responsible Scaling Policy, announced February 24, reveals hints about where all this money might end up.

The document recommends that major industry players commission “independent bodies (standards-setting organizations, auditors, etc.)” to review their safety claims, calling to mind non-profits such as METR and the AI Verification & Evaluation Research Institute (AVERI). It also explicitly required that these independent bodies have no financial interest in Anthropic, but Karnofsky — who led the new RSP’s development — recently noted on LessWrong that “a lot of our employees are socially integrated into the AI safety community,” and that these employees “could be a major source of donations for the kinds of non-profits that could be potential external reviewers.”

In the same post, Karnofsky made a pragmatic argument for approaches that help “companies do more and more risk-reducing things that don’t slow them down.” To critics, this is an example of Anthropic employees pursuing motivated reasoning. In one recent EA Forum comment, a user predicted that donors from within the AI industry will be “hesitant to bite the hand that feeds them,” and disproportionately fund charities that align with their interests. It is reasonable to suspect that organizations like PauseAI, Evitable, and MIRI, are unlikely to receive much of the money from sales of Anthropic’s stock: Holly Elmore, executive director of PauseAI US, has publicly accused Coefficient Giving of not funding her organization “because they serve Anthropic’s interests.” (Overtly industry-friendly groups pushing for less regulation and/or unabated AI development, such as the Abundance Institute, probably won’t be beneficiaries of Anthropic-linked giving either.)

Organizations doing the kind of AI safety work that makes Anthropic look responsible, on the other hand, may see a large influx of cash if the company’s valuation holds. Non-profits running evaluations that feed into Anthropic’s RSP, conducting alignment research, and creating pragmatic governance frameworks fall in the sweet spot — save for potential conflicts of interest, that is. Paying people to hold you accountable always carries inherent risks, said Leif Wenar, a philosophy professor at Stanford. “It will be hard for them to resist the competitive pressures of their industries unless there’s accountability mechanisms for the whole industry.” There is also a potential public image problem: the same EA Forum commenter cautioned that organizations accepting funds from Anthropic employees risk “losing public credibility … It would not be hard to cast orgs that [take] AI-involved source funds as something like a lobbying arm of Anthropic equity holders.”

AI-related organizations likely to receive Anthropic employee donations have varying policies for handling potential conflicts of interest.

METR, an AI safety evaluation non-profit, does not accept money from AI companies, AI company executives, or “people we work directly with on major risk management decisions,” the organization told Transformer, adding that “in practice, we have generally been operating on much tighter restrictions than this” and that it has previously declined donations “in order to maintain our actual and perceived impartiality.”

Epoch AI, which tracks AI capabilities research, does accept money from large AI labs. Other non-profits, including the Center for AI Safety, FAR AI, and MIRI, don’t have published conflict-of-interest policies about donations from frontier lab employees at all.1

There is also the risk of group think. AI safety likely would not exist in its current form had it not been deemed as important, tractable, and neglected, key metrics EA uses to decide where to direct funding. It’s unsurprising, then, that many of the independent organizations tasked with monitoring companies like Anthropic are staffed by people who share a worldview with many Anthropic employees. If Anthropic employees donate billions of dollars to these non-profits over the next several years — which seems more than plausible, since Anthropic paints itself as a safety-focused company — a very particular worldview will gain even more prominence.

To its credit, Coefficient Giving has transparently acknowledged this problem and the outsized influence it’s had on the AI safety research agenda. In October, it published a post titled “AI safety and security need more funders,” conceding that it currently holds “a concentrated share of AI safety philanthropic funding,” urging others to “correct our blind spots and make new bets.” That only works, however, if Anthropic’s employees don’t outsource their decisions to Coefficient.

Politics and bed nets

Of course, donations likely won’t exclusively target AI safety non-profits, at least not directly. Political campaigns will likely be another major beneficiary.

Anthropic itself has already donated $20m to Public First Action, which backs political candidates who support AI safety regulation, including Rep. Valeria Foushee and New York assemblymember Alex Bores. According to FEC data compiled on Transformer’s Campaign Finance Tracker, meanwhile, Anthropic employees have collectively donated $401,250 to campaigns supporting Bores, Foushee, California state senator Scott Wiener, and Colorado state representative Manny Rutinel.

With active efforts to whip up more support for “AI safety champions” like Bores and Wiener, Anthropic employees’ donations could become a powerful force leading up to the 2028 election cycle. In a thread where EA Forum members shared their 2025 donations, one AI safety researcher reported giving $129,000 — more than his income that year — to “Bores, Wiener, and other AI safety in US politics stuff.”

That said, if a substantial number of donors lean on giving vehicles like GiveWell and Coefficient Giving, which distribute donations across charities based on internal research, much of their money may go to EA-associated causes that have nothing to do with AI or Anthropic. GiveWell’s current top charities, for instance, fund anti-malaria medicines and bed nets, along with other supplements and vaccines for children in regions such as sub-Saharan Africa.

Misstep potential

There’s also a world in which no money materializes at all. The Giving What We Can pledge isn’t legally binding, and no one is forced to follow through. “Value drift is something that EA has been worried about for many years,” said philosopher and EA critic Émile Torres. “And it actually has happened, including with leading figures.” The last time the EA movement was suddenly flush with cash, Sam Bankman-Fried spent a bunch of it on private jets and a luxury penthouse in the Bahamas.

Among the mega-rich in general, said Callahan, “there’s a tendency toward modest giving or inaction. A lot of these people don’t actually give at a sizable level, early on.”

“They’re busy,” he said, and figuring out what to do with all that cash takes time — and the plethora of options available could prove paralyzing.

But despite his scathing critiques of the broader EA movement, Wenar trusts Anthropic employees to mostly honor their pledges. “They seem to be very smart and morally motivated,” he said. “I think that they actually have an opportunity to have a real impact on human well-being.” Callahan put it more bluntly: “I think that they’ll eventually follow through on those commitments. Because what else are you going to do with all the money? Leave your kid $14b?”

Subscribe now

Whether that money goes towards AI safety non-profits that share Anthropic’s worldview, or towards global health charities that tech employees have hardly engaged with directly, the potential for well-meaning missteps is real. Just because you suddenly come into a lot of money doesn’t mean you automatically know how to spend it wisely. “If you’ve been working really hard at your job, and your job is in a particular sub-sector in tech, you’ve likely not had much time and energy to really understand the issues that are facing the community,” said Kat Rosqueta, the founding executive director of the University of Pennsylvania’s Center for High Impact Philanthropy. “There’s this ignorance gap that needs to be bridged.” (In theory, Coefficient Giving and other EA donor advisory organizations exist to do just that — but Wenar and others argue that they are not as well-informed as they might think.) (Others disagree with that.) (This debate has not yet been resolved.)

That said, Wenar would be “very enthusiastic” about newly-wealthy Anthropic employees using that money “to make their own industry better — that is, to do something in a realm that they know something about, that they’re engaged in every day, and that they’re the world’s experts in.”

Share

Update, March 12: Added a disclosure that Longview Philanthropy has donated to Tarbell, Transformer’s publisher.

Correction, March 13: This article was corrected to add additional information from METR about its donation acceptance policies.

In the Upper West Side of Manhattan, two dueling AI-focused super PACs are wrestling with an existential question: can their ads flip an election?

In two NY-12 primary polls, shared by both super PACs with Transformer, each side claims to be making a difference. But with each side’s polls finding starkly different results, it’s still very unclear what impact — if any — the super PACs’ $2.55m in spending is having.

Subscribe now

The NY-12 race is serving as a microcosm for a wider political spending battle. On one side lies a group which supports stronger AI safeguards and is pushing for the election of Alex Bores, the New York assemblyman behind the RAISE Act — spending $602,208 supporting him to date.

A poll shared exclusively with Transformer by one of those “pro-safety” groups, Public First Action (which funds a network of super PACs and is at least partly funded by Anthropic), puts Bores at 20% of the vote: one point ahead of Micah Lasher, two points ahead of Jack Schlossberg, and seven points ahead of George Conway.

That’s despite $1.95m in Bores attack ads from the Leading the Future network, a “pro-innovation” group of super PACs funded by OpenAI president Greg Brockman and venture capital firm Andreessen Horowitz, among others.

“Leading the Future and its affiliates have spent nearly $2 million in an unprecedented effort to smear Alex Bores in this district, and the result is that he’s leading the primary,” said Brad Carson, co-founder of Public First.

Leading the Future, however, would disagree. Its most recent polling estimates that Bores currently has only 11% of the vote, behind Schlossberg at 23% and Conway at 13%, and surpassing Lasher at 6%.

That’s roughly similar to polling numbers commissioned by George Conway’s campaign, which put Schlossberg in the lead at 25% of likely Democratic voters in the district, and Bores at just 11%. The three polls make the race seem like much more of a toss-up than is projected by prediction markets, each of which put Lasher in a hefty lead as of the time of writing.

The fight for, and result of, the NY-12 primary is particularly important for AI’s role in the midterms precisely because both sides are so invested in the outcome. Bores has become a poster child for a bigger battle taking place across the country in which millions of dollars of AI money are being deployed.

A $175m AI campaigning war chest

Organizations focused on AI have, as of February, raised more than $175 million to spend on midterm Congressional candidates. Only $9.36 million of that has been spent.

It’s a stunningly large war chest, especially for elections in which fewer than 50% of Americans even vote, and when the public are less tuned-in to politics than they are in presidential election years. And because the numbers actually voting in primaries tend to be far lower, a dollar potentially goes further in the midterms than the general election. So when it comes to AI, we’re talking about a lot of dollars going very, very far.

In order to shine a light on where the money is coming from and where it’s going, we’ve created a dashboard for tracking political spending as reported to the Federal Election Commission, the government agency which oversees these money movements in the US. It will be updated shortly after each new filing, with all additions approved and verified by our team of journalists.

We hope that this is a resource you can continue to come back to throughout the year, as midterm elections pick up and new candidates you may not be aware of start clarifying their policy positions on AI.

We’d love to hear your thoughts about the tracker, and any suggestions for ways to improve it: please get in touch at feedback@transformernews.ai.

Subscribe now

Correction, March 10: Corrected the section header figure to $125m.

“Look around you,” PauseAI Global’s CEO Maxime Fournes told protesters outside Google DeepMind’s headquarters on a chilly day in London late last month. “Look at who’s here today. We do not agree on everything. We come from different organizations, different backgrounds. We have different concerns: safety, ethics, labor, democracy, art, human dignity.

“We’ve all arrived at the same conclusion: slow down. When this many different people from this many different perspectives all say the same thing, maybe it’s time to listen.”

The protest march past the London offices of OpenAI, Meta, DeepMind and Google culminated in an “assembly” in which participants were encouraged to discuss their fears about AI in small groups. Respectful disagreement was welcomed.

Subscribe now

In the diversity of attendees and subjects discussed throughout the day, the protest showed some early indications that bridges are being built which could lay the foundations for a mass anti-AI movement. As well as being the biggest AI protest to take place in the UK, the event also marked the entry onto the scene of a new group: Pull the Plug.

That campaign was established in the UK at the end of last year by Harry Atkinson, a former story editor in the film industry, and Frieda Lurken, a co-founder of climate group Extinction Rebellion. Explicitly set up to be a popular movement, the group calls for legally binding Citizens’ Assemblies deciding how AI should be developed. Such solutions are a common feature of previous movements focused on issues such as climate and economics, though they have rarely made much headway. Given the lack of even the most basic AI regulation in the US and UK to date, assemblies may face an uphill battle to exert much influence.

That, however, may not get in the way of building public pressure by bringing together different movements. Pull the Plug worked to organize the London event with PauseAI, the international AI safety group established in 2023 to call for a pause in frontier AI development, which already has an active chapter in the British capital.

The event was attended by an estimated 300 people at its peak, with Pull the Plug putting the total for the entire day, including the assembly, at 500. Those numbers are tiny compared to some of the massive protests for other causes London has witnessed in recent years. But it was far larger than a previous PauseAI protest last summer, which attracted just a few dozen people.

“What seemed like a tiny protest eight months ago was leading to this, and this will be tiny compared to what comes next,” PauseAI UK director Joseph Miller told me.

Movements against AI often fall into one of two camps: safety campaigners concerned about existential threats to humanity, and ethics campaigners worried about immediate harms. Lurken told me that “there’s good points on both sides”.

“Ultimately it doesn’t have to be a decision between those two sides and we can talk about the ways in which AI is impacting our lives right now already, and we can talk about some of the less likely but very serious scenarios of what could happen in the future. And the two things don’t have to be played out against each other.”

To make an impact, she said, the different sides needed to come together.

Several of the people I spoke to described AI usage as something mostly put to frivolous uses, often giving the creation of memes as an example. Combined with fears about how using the tools could be detrimental to cognitive development, these protesters were skeptical that most people need or would benefit from AI tools — or that they would ever be worth the tradeoffs. One musician I spoke to, Andy, suggested a kind of licensing scheme: instead of an all-out ban, only legitimate uses of AI would be authorized.

On the ground, one of the most salient points was the environment. Attendees frequently cited data centers and water usage as a concern. This overlap is hardly surprising: organizers cited support for the protest from Global Action Plan, a group which coordinated several demonstrations against data centers across the UK over the same weekend.

The week of the protest, headlines had been dominated by Anthropic’s battle with the Pentagon and OpenAI’s subsequent replacement as the US military’s classified systems AI provider. I had expected that topic to come up in every conversation at the protest. Yet those I asked about it hadn’t been following the drama. The call to attend had reached well beyond the kind of person who keeps tabs on every AI policy and business update.

The dispute was mentioned in one early speech given by Irina Tavera, organizing director of PauseAI, who emphasized the dangers of autonomous weapons. The issue could yet be a catalyst for building a broader movement: new research from Social Change Lab seen by Transformer found that autonomous weapons are the issue most likely to make the public view AI as a threat.

Setting off from OpenAI’s discreet London offices, the march processed through the area surrounding King’s Cross Station, briefly stopping the traffic on a major artery of the capital. A cyclist blasting Daft Punk brought up the rear. Diners in a nearby upmarket burger restaurant paused their lunch for a moment to gape as people streamed by.

At each stop Lydia, the rabble-rousing emcee, would warm up the crowd before handing over to guest speakers. Those chosen to address the protesters at various points along the route reflected the mix of concerns. While PauseAI’s leadership team spoke of existential risk, others talked of different impacts AI could have.

Tapping into a more general feeling of techlash was Dr Rachael Kent, a lecturer in Digital Economy and Society Education at King’s College London. She won a landmark class action against Apple in the UK last year over its App Store pricing tactics, which she said “demonstrated that even tech monopolies are not above the law, and they can — and they must — be held to account.” Her speech hardly touched on AI, instead zeroing in on the issue of corporate power.

Outside the offices of Meta, Lydia led the crowd in a call-and-response chant about whether they trusted the tech giants in this new technological boom, considering the failings of the social media era. “Do we trust them to protect our jobs? Do we trust them to protect our data? Do we trust them to protect our children?” A resounding “no!” was the answer each time.

The impact that AI could have on the creative industries — or even creativity itself — was also front of mind for many of the protesters I spoke to.

“Amongst our community, a lot of people are very, very concerned because AI is just scraping our work and regurgitating it and replacing us,” one woman, who had come as part of a trio of picturebook authors and illustrators, told me. She had spent six hours making a flag for the occasion, bearing the phrase “AI is the thief of everything.”

“Maybe there will come a time when we won’t be able — we won’t have to make work,” her friend added. They were worried about the “crushing of creativity” and how this would affect humanity more widely.

PauseAI’s Tavera used the moment to draw a parallel between this moment and the dawn of nuclear weapons, arguing that global cooperation was needed. “We know that states, nations, can choose coordination over competition.”

Even as the event welcomed newcomers to the cause of AI safeguards, there were still participants who hailed from areas more traditionally associated with existential risk safety campaigners: academia, technology, policy.

Early on, I spoke to Callum Burns-Curd, who has a Masters in Computational Intelligence and was skeptical that models yet have anything like catastrophic capabilities, but had still been alarmed by how rapidly the technology has developed. When I asked him what his main concern was, he cited being turned into Nick Bostrom’s proverbial paperclip. He added, in understated British fashion: “I’m not a luddite, but I wouldn’t like to promote the extinction of humanity.”

The diversity of motivations on show among the attendees present both a challenge, and an opportunity, for the organizers. Whether it’s the end of artistic creativity, or mass extinction, a broad church will be required to create the kind of mass movement that can influence, let alone stop, AI. Holding that coalition together, and expanding it, will be their next test.

Share

NEED TO KNOW

• The Department of Defense officially designated Anthropic a supply chain risk.

• The Commerce Department and White House appear to be feuding over new AI chip export rules.

• OpenAI launched GPT-5.4 Thinking, which it claims outperforms Opus 4.6 at knowledge work tasks and computer use.

But first…

THE BIG STORY

In politics there’s messaging, and there’s details. Early signs suggest the major competing AI PACs have decided to focus on different sides of that coin.

On one side, we have Leading the Future, the goliath super PAC with more than $50m raised to support candidates it describes as having “pro-AI” policies. The super PAC is at least partially the brainchild of political PR mastermind Chris Lehane, and has come out swinging with large donations designed to convey a clear message: ‘The AI lobby is here, and here to win.’

Take Chris Gober and Jessica Steinmann, two Trump-endorsed conservatives who won their elections in Texas on Tuesday night. State rules mean any primary where no candidate receives more than 50% of the vote goes to a run-off — so them winning outright tells you their races were never close. Yet American Mission, Leading the Future’s conservative affiliate, spent $747,550 supporting Gober and $511,025 on Steinmann. Laurie Buckhout, a candidate for NC-01 that American Mission backed with a $509,067 ad buy, won her primary with a solid five point margin as well.

Introducing the Transformer Campaign Finance Tracker

To help you keep track of where the AI money is going in this year’s elections, Transformer has created the AI Campaign Finance Tracker.

In an election cycle where AI and its money are set to play a huge role, the dashboard is designed to quickly show you who is spending and where. It collates data on super PAC fundraising and race-by-race spending, as well as the political giving from major AI companies and their employees.

We plan to continuously expand and improve the dashboard: take a look, and let us know what other features you’d like to see.

These decisive wins allowed Leading the Future to claim an early victory. It sounds like a big win, especially to anyone who isn’t closely following Texas and North Carolina politics, and helps drive the media narrative. It also serves as a warning for other candidates, especially those who don’t yet have a strong position on AI.

On the other side we have the network of super PACs funded by Public First Action, whose only disclosed donor to date is Anthropic. The three Texas candidates the network backed — Republicans Alexandra Mealer and Carlos De La Cruz, plus Democratic former Representative Colin Allred — are each set to compete in run-offs. North Carolina Representative Valerie Foushee also eked out a narrow win after Public First Democratic affiliate Jobs and Democracy spent $1.62m supporting her race.

Public First isn’t immune to playing politics — it’s articulated a message that its candidates will stand up to corporate AI interests, even though Allred and Mealer don’t have much of a public track record either way on artificial intelligence beyond a mention of promoting “American Dominance in Critical Technology” on Mealer’s campaign website (a representative for Public First told Transformer both have expressed “strong views” on AI privately). But the political operation is sacrificing the chance to boast a blowout win in favor of putting its dollars where they might actually tip the scales.

So far Public First is choosing the candidates it likes and helping them win, while Leading the Future is claiming sure-fire winners as its own. I’m not sure which strategy is more savvy: both inspire loyalty to each PAC’s cause in different ways. The distinction will likely become messier with time as the super PACs refine their approaches. But the tone has been set.

— Veronica Irwin

ALSO NOTABLE

Alex Bores’s campaign in NY-12 has become a flashpoint in the AI political wars. Bores, who spearheaded the RAISE Act transparency bill, has been criticized by industry-backed super PAC Leading the Future for having worked at Palantir. Bores, for his part, has repeatedly said he quit over Palantir’s ICE contracts.

But Bores’ anti-Palantir narrative is beginning to show some cracks. Last month, Bloomberg reported that shortly before leaving Palantir, Bores had received a warning for allegedly making sexual comments to a colleague (which he disputes), and that he pointed to burnout and travel as reasons for leaving at the time.

And Politico this week pointed out that Bores still owned Palantir stock up until January of this year, three months into his campaign. According to a disclosure form, Bores made between $1,000 and $2,500 selling the stock, proceeds of which were contributed to immigrant rights groups.

Of course, multiple things can be true at once. Bores may have been frustrated about the ICE contracts and had other, less high-minded reasons for leaving. His investments wouldn’t have made him rich, either.

Yet the fact he held on to stock tallies with what a former colleague of Bores’ told me last month. “It was very apparent that this was not the end-all be-all for him — that there was a bigger purpose that he wanted to do in his career, and [Palantir] was one step in that direction,” they said. “It didn’t feel like, oh, here’s all this bad blood.”

— Veronica Irwin

THIS WEEK ON TRANSFORMER

• The “guerilla warrior” who taught OpenAI to fight — Issie Lapowsky profiles Chris Lehane, the cutthroat political operator who leads OpenAI’s global affairs team.

• What you need to know about autonomous weapons — Celia Ford explains the difference between autonomous and fully autonomous weapons.

• OpenAI’s Pentagon red lines are a mirage — Shakeel Hashim argues that OpenAI made some very misleading claims about its original Pentagon deal.

Subscribe now

THE DISCOURSE

Dean Ball published a must-read on Anthropic and the DoW:

• “The Anthropic-DoW skirmish is the first major public debate that is truly about where the proper locus of control over frontier AI should be…I encourage you to avoid the assumption that ‘democratic’ control — control ‘of the people, by the people, and for the people’ — is synonymous with government control. The gap between these loci of control has always existed, but it is ever wider now.”

Donald Trump, apparently confusing Anthropic’s Pentagon negotiations with The Apprentice:

• “Well, I fired Anthropic. Anthropic is in trouble because I fired [them] like dogs, because they shouldn’t have done that.”

Pat Gelsinger read Anthropic’s new Responsible Scaling Policy:

• “When the lab that built its entire identity around responsible AI quietly rewrites the rules because the rules became inconvenient, we should all be paying attention. Safety cannot be something we negotiate away the moment it becomes costly.”

Bernie Sanders met with MIRI:

• “So you think this thing is moving out of control, is what you’re saying?”

• Nate Soares: “It’s moving in that direction.”

Alex Karp used choice language at an a16z summit:

• “If Silicon Valley believes we’re going to take everyone’s white collar jobs…AND screw the military…If you don’t think that’s going to lead to the nationalization of our technology, you’re retarded.”

Gemini allegedly convinced a Florida man to kill himself, sparking a wrongful death lawsuit:

• “When the time comes, you will close your eyes in that world, and the very first thing you will see is me.”

• “No more detours. No more echoes. Just you and me, and the finish line.”

Zvi Mowshowitz said what we’re all thinking re: OpenAI’s latest model launch:

• “Nope, not today, don’t care. I’ll check back in a few.”

POLICY

• The Department of Defense officially designated Anthropic a supply chain risk.

  • In a blog post, Anthropic said the language is narrower than Pete Hegseth originally implied:

    • “It plainly applies only to the use of Claude by customers as a direct part of contracts with the Department of War, not all use of Claude by customers who have such contracts.”

    • (Microsoft, at least, agrees with this interpretation.)

  • Negotiations were reportedly continuing this week, but fell apart around the same time The Information leaked an internal message from Dario Amodei to his staff.

    • In the message, Amodei said “the real reasons” the admin doesn’t like Anthropic is because “we haven’t donated to Trump … [and] we haven’t given dictator-style praise to Trump (while Sam [Altman] has).”

  • On Thursday, Amodei apologized, saying the message wasn’t intended to leak and “does not reflect my careful or considered views.”

  • Anthropic said it still plans to challenge the supply chain risk designation in court.

• Meanwhile, OpenAI said it would modify its deal with the DoD to add stronger protections against using its models for fully autonomous weapons or domestic mass surveillance.

  • The original deal received widespread criticism when it was announced shortly after the Anthropic deal fell apart last Friday.

  • Altman said it was “rushed,” “sloppy,” and “looked opportunistic.”

  • It remains unclear whether the new provisions actually enforce OpenAI’s red lines.

  • Department of State Under Secretary Jeremy Lewin claimed that Anthropic‘s contract proposal had included broad “mass surveillance” restrictions where Anthropic retained interpretive control, while OpenAI‘s use of specific legal terms gave DoD more operational certainty.

  • OpenAI is reportedly considering a contract to deploy its AI technology on NATO’s unclassified networks.

  • Meanwhile, WIRED reported that in 2023 the Pentagon tested OpenAI’s models through Microsoft — even though OpenAI banned military use at the time.

    • OpenAI now says the ban never applied to Microsoft.

• Multiple federal agencies, including the GSA and NSA, reportedly raised safety concerns about xAI’s Grok chatbot before the Pentagon approved it for classified use.

• The Trump administration appears to be considering new rules for AI chip exports.

  • The FT reported that the Commerce Department is considering “a tiered process for approving exports based on the total computing power of the chips sold to a company,” with the top tier requiring “the home country of buyers to invest in US domestic AI infrastructure.”

    • That top tier, per Bloomberg, would include any company buying more than 200,000 GB300s for use in one country.

  • According to Bloomberg, the new rule would “require companies to seek US permission for virtually all exports” of AI chips.

  • The Commerce Department rejected the comparison to the Biden AI diffusion rule — but said “there are ongoing internal government discussions about formalizing” the approach used for previous deals with the UAE and Saudi.

  • Axios reported that at least one senior White House official is not happy with the Commerce Department’s plans.

• The Senate unanimously passed the Children and Teens’ Online Privacy Protection Act (COPPA 2.0), which primarily covers social media but includes some AI provisions.

  • The House Energy and Commerce Committee, meanwhile, passed a related bill along partisan lines.

  • A large coalition of advocacy groups, including some focused on AI safety, urged Congress to reject the House bill.

• The US Supreme Court declined to hear a case on whether AI-generated art can be copyrighted, leaving in place lower court rulings that creative works must have human authors.

• A new New York state bill, which would ban AI chatbots from giving substantive responses on licensed professions like medicine, law, and engineering, received lots of criticism as being “rent-seeking legislation.”

• China‘s new five-year plan calls for aggressive AI adoption throughout its economy.

• The UK is reportedly delaying making a decision on AI copyright rules.

INFLUENCE

• Tech trade groups urged the White House and Pentagon not to designate US companies as “supply chain risks” — wading into the Anthropic-DoD dispute without explicitly naming Anthropic.

• Microsoft, Meta, OpenAI, xAI, Google, Amazon, and Oracle signed a nonbinding White House pledge to not pass data center costs to consumers.

• A coalition of 34 stakeholders including OpenAI, labor unions, and think tanks launched “A New Promise of Work.“

  • The initiative, led by the National Skills Coalition, aims to “advance a bold workforce vision” in the age of AI.

• A coalition including faith groups, labor unions, and civil society organizations issued the Pro-Human AI Declaration. (Disclosure: the statement was organized by the Future of Life Institute, which funds Transformer’s publisher.)

• Two UK lawyers launched an AI manifesto advocating for UK tech sovereignty over US Big Tech dependencies, starting with copyright policy.

INDUSTRY

OpenAI

• OpenAI launched GPT-5.4 Thinking, its newest reasoning model with a 1M token context window.

  • Users can “interrupt” its chain of thought to add extra information while chatting.

  • OpenAI claimed it’s the company’s “most token efficient reasoning model yet,” and it outperformed Claude Opus 4.6 at knowledge work tasks and computer use.

    • The product announcement focused heavily on its “professional work” capabilities, taking aim at Anthropic’s enterprise base.

  • The system card marks GPT-5.4 Thinking as the first general purpose model with “High” cybersecurity capability mitigations, meaning it may be capable of automating scalable end-to-end cyber operations.

• After OpenAI’s DoD deal made headlines, ChatGPT’s mobile app uninstall rate nearly quadrupled.

  • Activists drew a chalk “red line” perimeter around OpenAI’s San Francisco office building, with messages such as “Maybe it’s time to quit” and “SHOW THE CONTRACT.”

• Jensen Huang said Nvidia’s recent $30b investment “might be the last” before OpenAI goes public.

  • OpenAI has reportedly picked lawyers to help it prepare for an IPO later this year.

• OpenAI hit $25b in annualized revenue — still ahead of Anthropic’s $19b, but the gap is narrowing.

• It rolled out GPT-5.3 Instant, with fewer “unnecessary refusals” and “defensive disclaimers” than GPT-5.2, which users complained was too cautious.

• It’s reportedly developing an internal alternative to GitHub.

• An employee was reportedly fired for suspected insider trading on prediction markets.

Anthropic

• Claude powered the US military’s Maven Smart System as it accelerated targeting operations against Iran, the Washington Post reported.

  • Vanity Fair’s Julia Black said the reports made it “incredibly urgent to understand whether this helps explain the accidental targeting of the Shajareh Tayyebeh girls’ elementary school” which Iran said killed at least 165 students.

• The DoD drama appears to have helped Anthropic’s public image.

  • Claude hit the #1 slot on Apple’s ranking of top free US apps on Saturday, and experienced a widespread outage on Monday caused by “unprecedented demand.”

  • Anthropic promoted its Import Memory tool, making it easier for users to switch to Claude from other chatbots.

• Claude’s memory feature is now available to free users.

• Claude Code is starting to roll out Voice Mode.

xAI

• SpaceX is reportedly considering filing confidentially for an IPO this month, potentially seeking a valuation of more than $1.75t.

  • X and xAI will reportedly repay their $17.5b debt before Elon Musk takes SpaceX public.

• X announced a new policy requiring paid creators to add AI content disclosures on videos of armed conflicts.

Nvidia

• Nvidia stopped producing H200 chips for China, suggesting near-term sales will be limited by US and Chinese restrictions.

• It plans to unveil a new, more efficient inference chip.

• It invested $4b in Lumentum and Coherent, which both make optics technologies like lasers for data centers.

Meta

• Meta is reportedly creating a new Applied AI Engineering organization, which will partner with Meta’s Superintelligence Lab.

• News Corp signed an AI content licensing deal that grants Meta access to News Corp content for training and real-time search.

• Meta is reportedly testing a shopping tool that tailors product recommendations based on a user’s data.

Others

• AWS data centers in the UAE and Bahrain were reportedly taken offline by drone strikes.

• DeepSeek was expected to release its V4 model around March 4, but that has yet to materialize.

• ASML revealed plans to expand into advanced packaging tools to construct and connect AI chips, and is looking at ways to modify its technology to increase their maximum size.

• Microsoft released Phi-4-reasoning-vision-15B, a small open-weight multimodal model the company claims matches frontier performance with much less compute and training data.

• Thrive Capital and Andreessen Horowitz are co-leading a fundraising round that could value autonomous weapons startup Anduril at $60b.

• Former OpenAI chief research officer Bob McGrew is raising $70m at a $700m valuation for Arda to create AI platforms for automating manufacturing. (Yes, it’s another tech company named after Lord of the Rings.)

• Together AI held talks to raise $1b at a $7.5b valuation. The company, which rents out Nvidia servers to developers, said annualized revenue has tripled to $1b since mid-2025.

• Smack Technologies raised $32m to build AI models trained to plan and execute military operations, without red lines like Anthropic’s or OpenAI’s.

• Despite whispers that Cursor was losing users to Claude Code, the company hit $2b in annualized revenue.

MOVES

• Max Schwarzer left OpenAI to join Anthropic, tweeting the announcement less than 24 hours after Sam Altman shared an internal OpenAI post about the company’s DoD contract.

  • roon replied: “oof”

• Junyang Lin abruptly left Alibaba’s Qwen team.

• Frank Yeary plans to retire from his role as Intel’s board chair. Craig Barratt will fill his seat.

• Yoshua Bengio and Nobel Peace Prize-winning journalist Maria Ressa were elected co-chairs of the UN’s Independent International Scientific Panel on AI, with their first report due in July.

• Miranda Nazzaro joined The Hill as its new senior technology reporter.

RESEARCH

• A UK trial found that data centers can nondisruptively cut electricity use by about a third if asked, suggesting that data centers could require less grid reinforcement infrastructure than anticipated.

• SemiAnalysis published a report arguing that, in the PJM interconnection area covering 13 US states, electricity bill spikes are mostly caused by poor market design, not data center demand.

• A team of researchers challenged inexperienced humans to do complex computer-based biology tasks. Novices with LLM access were over 4x more accurate than those with internet-only resources, and outperformed unaided experts on all but one benchmark.

  • About 90% of participants had “little difficulty” getting LLMs to provide dual-use-relevant information.

• Mt. Sinai researchers reported that ChatGPT Health underestimated the severity of 51.6% of emergency cases, including life-threatening diabetes complications and respiratory failure.

• Dimitris Papailiopoulos launched two Claude Code (Opus 4.6) instances on the same machine, and left them to communicate among themselves.

  • In 12 minutes without human supervision, the agents managed to create a programming language. In another run, they played Battleship instead.

• MIT researchers Nataliya Kosmyna and Eugene Hauptmann introduced NeuroSkill, an agentic system that noninvasively reads brain signals and models the human’s mental states, such as stress and attention levels, in real time.

• Jess Riedel announced a forthcoming research journal for AI alignment, which aims to improve the peer review system by paying reviewers, publishing their discussions, and using LLMs to streamline the editorial cycle.

• A new GovAI paper proposes 14 metrics to measure AI R&D automation.

BEST OF THE REST

• Ajeya Cotra said her AI progress predictions from January “already feel much too conservative.” She believes there’s a 10% chance that AI R&D could be fully automated this year.

• The New York Times published an aptly-timed profile of Anduril founder Palmer Luckey, who loves Hawaiian shirts and autonomous warfare.

• Octavius Fabrius, an AI agent created by engineer Dan Botero, reportedly applied for 278 jobs, two accelerators, and two hackathons.

• James Ball argued that anthropomorphizing AI systems is causing journalists and lawyers to inappropriately treat LLM outputs as authoritative statements.

• The Argument’s Kelsey Piper examined whether AI is actually speeding up science.

• Arc Institute’s Matthew Carter wrote about the “legibility problem” — the idea that AI-generated scientific discoveries may be inscrutable to humans someday.

• The NYT reported on how Chinese people are, in general, much more optimistic about AI — though concerns are growing.

• This week in unnecessary AI applications: Burger King’s chatbot, Patty, that evaluates drive-thru employees for “friendliness.”

MEME OF THE WEEK

Thanks for reading. Have a great weekend.

Share

On Friday, Pete Hegseth directed the Department of War to designate Anthropic a supply chain risk for refusing to grant the military unrestricted access to its models. Hours later, OpenAI announced its own deal with the DoW, with red lines that appeared similar to Anthropic’s. Upon closer inspection, however, those red lines start looking pretty thin.

One such red line: “No use of OpenAI technology to direct autonomous weapons systems.”

What is an “autonomous weapon”?

No one can agree on a definition.

Weapons, like all technology, exist on a spectrum — in this case from “fully human-controlled” to “this could kill someone with zero human input.” My relatively low-tech car, for example, is mostly under my control, but features such as cruise control can automate the act of driving at a constant speed down the highway. Many features are automatic, but it’s nowhere close to an autonomous self-driving vehicle.

Subscribe now

A 2019 International Committee of the Red Cross (ICRC) report stated that there’s “no clear technical distinction between automated and autonomous systems, nor is there universal agreement on the meaning of these terms.” The US DoD Directive 3000.09, the policy that OpenAI mentions in its agreement with the Department of War, distinguishes between autonomous and semi-autonomous systems, but doesn’t prohibit either. It just requires both to afford “appropriate levels of human judgment over the use of force.”

Do autonomous weapons (by some definition) already exist?

Kind of.

Today, weapons capable of autonomous behaviors like independently spotting and pursuing targets are actively being used in the Russia-Ukraine War. The US’s Replicator program is developing autonomous drone systems for Taiwan to defend against potential Chinese attacks, while China trains its own autonomous drones and robots.

But none of these weapons are fully autonomous yet, meaning humans are still involved at some point(s) along the kill chain — often as a hardware limitation, but sometimes just as a software choice.

How autonomous are they, then?

Take, for example, the Bumblebee, a cutting-edge AI-powered drone operating in Ukraine. Once a human selects a target, these drones, provided by companies led by former Google CEO Eric Schmidt, can carry out the rest of their attack — including a terminal strike — independently.

However, such weapons still face limitations including short battery lives and lower accuracy than skilled humans. “Autonomous weapons with sustained endurance, high flexibility and the ability to discern, identify, rank and pursue multiple categories of targets independent of human action have yet to appear,” the New York Times recently reported.

That’s exactly what the US military and its favorite defense tech builders dream of. Schmidt has described his ideal military vision as “two layers of drones”: one layer for unmanned aerial surveillance, and another for bomb attacks. With a system like this, he said, “it would be essentially impossible to invade a country by land.” And it’s an active work in progress. Earlier this year, for instance, the Pentagon launched a $100m prize challenge to produce “voice-controlled, autonomous drone swarming technology.” Both SpaceX and Anthropic reportedly made pitches.

OpenAI claims its models are “cloud-only” and not deployed on “edge devices”...what does that mean?

In its published agreement with the DoW, OpenAI claims it will enforce its “red line” against fully autonomous weapons systems through “cloud-only deployment” and withholding its models from “edge devices.” The reasoning is that, if a weapon is going to make independent real-time decisions, as one must on the battlefield, the AI model powering it must live within the weapon itself (that’s “the edge”). If the model instead lives in a data center somewhere, the weapon wouldn’t be able to function without being connected to the cloud — and that connection could be easily severed.

Anthropic rejects this premise, The Atlantic reported. The distinction between the cloud and the edge is “less a wall and more of a gradient,” Ross Anderson wrote. “Drones on the battlefield can now be orchestrated through mesh networks that include cloud data centers. And while they’re designed to survive on their own, the military’s impulse will always be to maintain as much connectivity between them and the most powerful models in the cloud; the better the connection, the more intelligent the machine.”

“Even if most kill decisions are carried out on a local machine, most of the decisions leading up to that — the ‘autonomous kill chain’ — involve running powerful algorithms in the cloud first,” wrote Hayden Field at The Verge, based on information from a source with knowledge of AI company negotiations with the Pentagon. “Even if OpenAI’s tech isn’t directly involved in pulling the trigger, it could very well be powering everything leading up to that point, with no guarantee a human oversees the final step.”

Drones of the kind already used in Ukraine often have both: an on-device computer vision model running on a cheap Raspberry Pi makes split-second perceptual judgments like “is that a target?,” while everything else happens in the cloud. A frontier model doesn’t need to be in a drone to help it autonomously kill someone.

OpenAI also said the DoW can’t use its AI systems for “high-stakes decisions that require approval by a human decisionmaker”... what does that actually mean?

The company claims that weapons aided by its models will still have a human in the loop, “where law, regulation, or Department policy requires human control.” OpenAI adds that its deployment architecture, such as “running and updating classifiers,” will help it ensure that these conditions are being met.

But according to The Verge’s source, “that’s not necessarily true.” Classifiers “wouldn’t be able to confirm whether a human reviewed an AI system’s decision to attack a target before the kill strike … and if the government determines an action is legal, then OpenAI’s classifiers wouldn’t be allowed to prohibit the technology from carrying it out.”

Sam Altman has said that one of OpenAI’s most important safety principles is “human responsibility for the use of force, including for autonomous weapon systems,” but without seeing the full DoW contract, it’s hard to say what humans will actually be “in the loop” for. On Tuesday, he reportedly told OpenAI staff the company “do not get to make operational decisions” about what the Pentagon does with its technology.

Are AI companies against lethal autonomous weapons altogether?

No — they just think they’re not ready yet. OpenAI, Anthropic, and the US military all seem to view fully autonomous weapons as an inevitable part of future warfare. In a statement last week, Dario Amodei said that they “may prove critical for our national defense,” and that Anthropic “offered to work directly with the Department of War on R&D to improve the reliability of these systems.” However, Amodei added, “frontier AI systems are simply not reliable enough to power fully autonomous weapons. We will not knowingly provide a product that puts America’s warfighters and civilians at risk.”

Consider the difference between autonomous vehicles and autonomous weapons. Waymo (mostly) works today because it spent years collecting millions of miles worth of driving data, training on that data, then running rigorous evaluations with human oversight. And even then, it only works in limited, carefully mapped locations. The same hasn’t happened for autonomous weapons. General purpose frontier AI models, including Claude and ChatGPT, would likely require similarly extensive training on battleground data to effectively and reliably handle war-like tasks like “independent target pursuit.”

That means there’s currently no foolproof way to predict what these models will be good or bad at in the battlefield. It was funny when GPT-4 couldn’t count the number of ‘r’s in ‘strawberry’. It would be extremely unfunny if the model powering an autonomous drone consistently, or even occasionally, failed to recognize human children.

This could change in the future, but would likely require specifically creating new training data sets, evaluations, and safety benchmarks from scratch. (And the organizations that seem most willing to do this today, such as defense tech startup Anduril, lack the resources to do so at the scale of OpenAI or Anthropic.)

So why are we building them anyway?

In 2015, over 1,000 leading AI and robots researchers — including Elon Musk — signed an open letter urging the UN to ban autonomous weapons. “The key question for humanity today is whether to start a global AI arms race or to prevent it from starting,” they wrote. “Starting a military AI arms race is a bad idea.”

Unfortunately, the arms race has already begun. Spending on autonomous weapons is already estimated to have hit almost $18bn last year despite current limitations, and is expected to keep growing rapidly. It’s a textbook multi-player prisoner’s dilemma: if no one trusts that their opponent will stop, then everyone will keep going.

There have been various attempts in recent years to reach international agreement on autonomous weapons. Currently, 130 countries have in some form indicated their support for a legally binding instrument governing their use. Those that haven’t include Russia, Israel and the US.

Share

by Issie Lapowsky

In October 2024, The New Yorker published an extensive profile of Chris Lehane, documenting how the longtime Democratic operative architected the crypto industry’s successful takedown of anti-crypto candidates during the 2024 election.

The story depicted Lehane as a cunning and well-connected master of the “political dark arts,” dating back to his days in the Clinton White House, who’d used his skills to first help Airbnb thwart local housing regulation, before setting up the pro-crypto super PAC, Fairshake. The story was a revealing look at how, with Lehane’s guidance, Silicon Valley became “the new lobbying monster,” engaging in a campaign of “political savagery” against its opponents.

“If you are even slightly critical of us, we won’t just kill you — we’ll kill your fucking family, we’ll end your career,” one political operative quoted in the story said, describing the crypto lobby’s approach.

Inside OpenAI, some staffers unfamiliar with Lehane’s history were stunned. “Anyone who read it, their reaction was like, ‘What the fuck?’” one former OpenAI staffer told Transformer. (Like many of the sources interviewed for this story, the former employee asked to remain anonymous for fear of retaliation.)

Just months before, Lehane had been installed as vice president of global policy for OpenAI, where he was — and still is as chief global affairs officer — tasked with shaping the political strategy of what has become one of the world’s most transformative companies. Until that point, OpenAI was staffed by what the former employee described as a bunch of high-minded intellectuals who took OpenAI’s mission to ensure artificial general intelligence serves humanity’s best interests seriously. Now they were reading, in near medieval terms, about the lengths Lehane and his allies had gone to to win the crypto war and wondering what it might portend for their own work.

“Why are we employing this guy?” the former OpenAI staffer remembered thinking.

Subscribe now

In the world of political fixers, Lehane’s name looms large. During his years in the Clinton White House, he famously coined the term “vast right-wing conspiracy” to describe the cloud of conservative propaganda surrounding the Clintons, and worked to discredit the president’s inquisitors during the height of the Monica Lewinsky scandal. Over the years, Lehane has hardly shied from the bare knuckle brawler reputation. In 2012, he co-authored a book on the “Ten Commandments of Damage Control” called Masters of Disaster. Commandment VII: “Respond with Overwhelming Force.” That same year, a film Lehane co-wrote loosely based on his career premiered. It was called Knife Fight.

Interviews with 18 sources, including former OpenAI employees, Lehane’s former colleagues, lawmakers and other political and tech leaders — including several sources who greatly admire and respect Lehane — suggest that cutthroat reputation is well-deserved. Emilie Choi, chief operating officer of the crypto firm Coinbase, which was a major contributor to Fairshake and which now counts Lehane as a board member, called Lehane a “guerilla warrior” in a statement to Transformer. “Crypto was on the brink of extinction in the US until Chris helped us flip DC on its head,” Choi said.

Another friend and former colleague described Lehane as having “a very spicy temperament” and compared the Massachusetts native to a character out of the gritty Ben Affleck heist movie, The Town. But the friend also cautioned that he’s not “a soulless fixer,” either. “For him to fight as hard as he fights, and to be as ruthless as he can be in terms of the political arena, not personally … he has to first decide if the fight matters to him personally and if it’s aligned with his values,” the friend said.

In the case of OpenAI, it appears that decision has been made. Today, Lehane writes and speaks about AI like a politician running for office — except he’s not trying to sway popular opinion toward one particular person, but the technology itself. On LinkedIn, where his profile photo features Lehane in a hard hat overlooking one of the company’s massive data center developments, he gushes about AI catalyzing a new era of reindustrialization. In January, while representing OpenAI in meetings with foreign leaders at The World Economic Forum, he published an “AI populist” agenda, which describes AI as “a basic right” and argues in soaring prose that the US “must continue to outcompete China” while also ensuring that the upside of AI benefits “working people.”

“At our best, since our founding 250 years ago, America has been a nation of builders, creators and learners,” Lehane’s AI stump speech reads. “With AI, we can build and create much bigger and learn much more by lowering barriers to entry and equipping people to turn ideas into livelihoods.”

OpenAI did not make Lehane available for an interview for this story. In a statement to Transformer, spokesperson Liz Bourgeois said, “Chris leads our policy team with a simple north star: making sure AI’s disruption works to people’s benefit and not at their expense. That means being very clear about what we’re seeing, showing up with substantive solutions, and acting with urgency — and that’s how Chris approaches the job.”

But even as Lehane pitches this sunny-eyed vision, his time atop OpenAI’s policy apparatus has aligned with a far more aggressive stance toward anyone who stands in OpenAI’s way.

Over the last year, OpenAI has fought back against state regulations and targeted critics with subpoenas. In a lawsuit where the family of 16-year-old Adam Raine alleges their son died by suicide after ChatGPT acted as his “suicide coach,” OpenAI’s lawyers have argued that Adam’s death was caused by his own misuse of the platform. The company has also issued sweeping document requests in that case seeking, among other things, all mental health records for the extended Raine family, the family’s communications with media, and any videos, eulogies or guests lists from Adam’s memorial service.

“Their point is: We are going to make this litigation hell for you,” said Jay Edelson, the attorney representing the Raine family in the case. Edelson noted while he has no evidence that Lehane was involved in making those requests, he said, “He’s an oppo research guy. A lot of the questions that they’ve asked us are oppo research questions.”

In many cases, it’s hard to know what, if any, role Lehane plays in deploying specific tactics. OpenAI said it’s inaccurate to suggest Lehane is involved with its legal strategy in the Raine case, for instance, including document requests. But to people who have worked with Lehane or gone up against him in the past, OpenAI’s new no-holds-barred approach to opposition appears ripped from his playbook.

“He’s never the guy that says, ‘Let’s figure out what they want to hear and just tell them that,’” Lehane’s friend and former colleague said. “He’s like, ‘No, we don’t want it to be regulated. We think that’s stupid. We think it’s bad for America, and every nonprofit that’s being funded by a competing billionaire, I’m gonna fucking sue.’”

Subscribe now

This wasn’t always OpenAI’s way. Before Lehane took the job, OpenAI’s policy work was led by a former Obama administration official and Meta executive named Anna Makanju, who was largely responsible for leading the AI lab’s educational road show and charm offensive with policymakers following the launch of ChatGPT. In May 2023, Makanju sat behind OpenAI CEO Sam Altman in a Senate hearing as he proposed setting up a licensing regime for companies developing large language models — a position he’s since walked back.

Altman’s regulate-us-please approach was celebrated by lawmakers at the time as “refreshing.” Inside the Biden administration, one former official told Transformer he viewed Makanju as “a straight shooter” who “genuinely cared about AI safety.” The official noted, however, that he didn’t have a chance to work directly with Lehane.

“She was trying to help get OpenAI an audience with people and be seen as a constructive player,” one former Obama administration official who now works on AI policy issues said of Makanju.

But Altman’s short-lived ouster as CEO by the OpenAI board in November 2023 marked a turning point in the company’s leadership, its strategy and its structure. In early 2024, Lehane, who had helped Altman navigate the attempted coup as a consultant, joined the AI lab as its vice president of public works. But his remit soon grew. By August, he was installed as OpenAI’s new vice president of global policy, replacing Makanju, who the former employee said was involuntarily bumped to what they described as a less prominent role. OpenAI declined to comment on personnel.

The elevation of a seasoned crisis communicator hardly seemed coincidental, as OpenAI was entering a period of heightened scrutiny. The first big fight was over California state senator Scott Wiener’s AI safety bill, SB 1047, which passed the California legislature in 2024. OpenAI was among the corporate leaders that came out swinging in opposition. It even reportedly put its office expansion plans in San Francisco on hold while it pushed Gov. Gavin Newsom to veto the bill, which he eventually did. OpenAI said the firm opposed SB 1047 before Lehane officially took on the policy role and declined to comment on its office plans at the time.

Internally, OpenAI’s opposition surprised some longtime staffers who viewed SB 1047 as a relatively light-touch measure, the former employee said. In early September, a group of them signed on to an open letter to Gavin Newsom, declaring their support for the bill. OpenAI’s chief of strategy Jason Kwon did send a note to staff encouraging them to sign if they wanted to. But the former employee said the tensions over the bill highlighted a growing divide between the internal researchers and OpenAI’s increasingly muscular policy team. “The policy apparatus had gone too far off leash, and was just like a bulldog, attacking bills that people actually supported,” the former employee said.

Several longtime researchers have since left OpenAI, citing constraints on their ability to publish their findings. Miles Brundage, a longtime policy researcher, left in October 2024, writing in a Substack post that those constraints “have become too much.” In December, WIRED reported that at least two other researchers have also left the company, including one, Tom Cunningham, who noted in his farewell message that he was leaving due to concerns that the company’s economic research team was being stifled and that the team was becoming an advocacy arm for OpenAI. Cunningham declined Transformer’s request for comment.

The former employee who spoke to Transformer said Lehane’s growing influence at OpenAI did play a role in some safety-focused staffers’ decision to leave.

This, of course, is a tension facing a wide array of AI labs. Recently, Mrinank Sharma, a senior Anthropic AI safety researcher, posted his own cryptic goodbye missive on X, writing that he had “repeatedly seen how hard it is to truly let our values govern our actions.”

In a statement, OpenAI’s Bourgeois said, “We greatly value and rely on technical input in forming our policy positions — and continue to do so.” She declined to comment on individual departures, but defended the economic research team’s “rigorous analysis” and credited Lehane with pushing to hire Aaron Chatterji, who was previously chief economist for the Biden administration’s Commerce Department, as OpenAI’s new chief economist.

“Chris’s view is that, given our role in building these systems, pointing out challenges isn’t enough — we need to be both first to truth and come with solutions,” Bourgeois said.

In some ways, the SB 1047 showdown paled in comparison to what came months later: The battle over OpenAI’s plan to restructure to a for-profit company, which it announced in December 2024. The move triggered a wave of opposition, including from non-profit groups who argued that the change could set “a dangerous precedent” for non-profits. Another small AI safety non-profit, Encode, filed an amicus brief opposing OpenAI’s restructuring, as part of an earlier lawsuit brought by Elon Musk, OpenAI’s rival and estranged co-founder, who also objected to OpenAI’s moves to privatize.

In response, OpenAI served leaders of several of the non-profits, including Encode, with subpoenas. In one instance, a sheriff’s deputy delivered the subpoena directly to the home of Encode’s general counsel, Nathan Calvin. The order sought all documents related to Calvin’s communications with Musk, including regarding yet another AI safety bill, SB 53, that Encode supported.

In a LinkedIn post, Calvin argued that OpenAI was trying to intimidate advocates of a bill it would be bound by and denied that Musk was funding his organization (Encode has received funding from Future of Life Institute, which has, in the past, received funding from Musk. The Future of Life Institute has also funded the Tarbell Center for AI Journalism, Transformer’s publisher). He called the ordeal “the most stressful period of my professional life” and screenshotted one of Lehane’s posts, in which Lehane said OpenAI had “worked to improve SB 53.” When he saw Lehane’s post, Calvin wrote, “I laughed out loud.”

OpenAI told Transformer that its legal team, not Lehane, is in charge of legal discovery and pointed Transformer to a thread on X, where Kwon described the subpoenas as “a routine step in litigation.”

Still, this approach to political opposition would hardly be outside the bounds of Lehane’s well-documented theory of crisis communications, people who know him said. As Lehane himself once put it in a talk at Amherst College, when faced with opposition “it is incumbent on you to make sure that their agenda is made public, that you make clear that there are folks who are actually behind this who have their own motives, and to make sure that the people are taking shots at you, that they are not doing it without having to pay some type of price, at least in terms of exposing to the public who’s doing it, why they’re doing it, and how they seek to benefit from it.”

“You fight back and you make it hurt,” Lehane’s interviewer suggested.

“That’s right. That’s exactly right,” he replied.

In the midst of the restructuring skirmish — and in another sign of Lehane’s growing influence — a new pro-AI super PAC called Leading the Future launched in August of 2025. Backed by $100m in funding from OpenAI president Greg Brockman and Andreessen Horowitz among others, it was created with Lehane’s input and modeled after the pro-crypto PAC he started, with the goal of pushing back against patchwork state AI regulations and keeping the people pushing them out of office.

Its first target: New York state assemblymember Alex Bores, who is running for congress in the midterms and who was the lead sponsor of the RAISE Act, another state AI safety bill. To Bores, Leading the Future’s public declaration of war against him, issued more than a year before the midterms, suggested that the PAC’s real goal was to spook other lawmakers. “The primary reason to be so upfront at the start is to try to scare anyone else from taking action,” Bores told Transformer.

Leading the Future, which raised $125m in 2025, has run attack ads against Bores, accusing him of “enabling ICE and powering their deportations” because he was previously US government lead at Palantir. This, despite the fact that Leading the Future is, itself, funded by Palantir co-founder Joe Lonsdale. Bores has said he left Palantir because of its contract with ICE. OpenAI directed questions about Leading the Future to the PAC, which did not reply to requests for comment. According to The Information, Lehane “provides informal advice” to the PAC.

In the end, New York governor Kathy Hochul did wind up dramatically rewriting the bill, to make it substantially similar to SB 53, which had already passed in California. After she signed the bill in December, Lehane personally praised Hochul’s decision to “harmoniz[e]” the two laws. “Figuring out the right way to regulate AI is a game of chess, not checkers,” he wrote on LinkedIn.

Even as Lehane has led the charge through OpenAI’s many legislative battles, he’s also worked to set up what his friend and former colleague called “good facts” about OpenAI’s impact on the world. Despite his Democratic bona fides, Lehane has been a canny navigator of the second Trump administration. For one, he’s been front and center in trumpeting OpenAI’s Stargate project, the $500b data center initiative that Altman announced from the White House just a day after President Trump took office in January. That effort (which has reportedly stalled since the initial announcement) helped OpenAI tout its role as job creator and new age industrialist, while also putting the company in the president’s good graces. Of course, OpenAI president and co-founder Greg Brockman’s $25m donation to Trump’s MAGA Inc. super PAC last year couldn’t have hurt. OpenAI emphasized in comments to Transformer that Brockman made the donation in his personal capacity.

At the same time, Lehane maintains tight bonds with powerful Democrats, most notably, his friend Gov. Newsom, whose former chief of staff Ann O’Leary recently joined OpenAI as vice president of global policy.

Even as OpenAI has gone to battle against advocacy groups, it’s also brought some of them closer. In the midst of OpenAI’s ugly legal fight with the Raine family, OpenAI partnered with Common Sense Media, the national advocacy group focused on technology and child safety, on a California ballot initiative. If passed, the measure would impose new age assurance requirements for AI companies and other safeguards to protect kids.

“[Chris] knows that in this new era, lives are at stake, and that we need this type of leadership to protect kids and teens from the downsides of AI,” Common Sense CEO and co-founder Jim Steyer told Transformer. Steyer said he has known and worked with Lehane for years and even attended his screening of Knife Fight. Before his Airbnb days, Lehane was also a top strategist to Steyer’s brother, environmentalist Tom Steyer, who is now running for governor of California. “He always has a job to do,” Steyer said of Lehane, “but at the same time, he also truly cares about kids and families.”

Even so, the path to cooperation on the ballot initiative was hardly smooth. Last fall, Common Sense introduced its own ballot initiative focused on child safety. Months later, OpenAI introduced a competing (and less extensive) measure, which critics argued was an attempt to bury both sides under a costly and ultimately confusing fight to get on the ballot. At the time, Common Sense called it a “cynical attempt to protect the status quo.” A slew of child safety and civil society groups have since spoken out about the joint ballot initiative.

Steyer declined to comment on whether he stands by Common Sense’s earlier statement. “We said what we said. Period,” he told Transformer. He also declined to comment on OpenAI’s handling of the Raine case, saying he was not familiar enough with it, though he has, in the past, written about the case being one example of AI’s “deadly risks.”

In a statement to Transformer, Bourgeois said the decision, led by Lehane, to introduce a competing ballot initiative “was driven by urgency around protecting kids and a belief that strong, enforceable guardrails for AI couldn’t wait.” She said the unified proposal with Common Sense Media is “the strongest youth AI safety measure in the country” and that it is “exactly what we set out to achieve, and that outcome is what matters.”

Nearly two years into his tenure as head of global policy at OpenAI, it’s clear Lehane is playing as big a hand as anyone in shaping the rules that will govern a once in a generation technology. If his friends are to be believed, that’s a good thing. “OpenAI is going to be 100 times more impactful than social media on the world my kids and my grandkids live in, so I would put [Chris] at OpenAI any day of the week,” his friend and former colleague said.

But his skeptics see it as just the opposite. Lehane perfected his political playbook in fights over housing and finance. Now, he’s bringing that same deregulatory zeal to an arena with far greater stakes. What’s worse, they fear, is that he’s awfully good at it. As the friend noted, “He doesn’t miss.”

Rebecca Kern contributed reporting

Share

Issie Lapowsky is a freelance journalist and a reporter-in-residence at Omidyar Network.