1/ Hundreds of contractors posed as teenagers to test how rival chatbots like ChatGPT and Gemini handle prompts about suicide and self harm. The project was run for Meta. buff.ly/Auslrgh

So thrilled for Karolina Stańczak, presenting our paper at FAccT today. You cannot write a complete contract for an AI, and we never wrote one for ourselves either. Norms, markets, and law fill that gap, and the paper asks what alignment could learn from them. Read here: arxiv.org/abs/2503.00069

Most AI safety work tests one model at a time. Put many agents together and you get cooperation but also risks like collusion and cascades. A new Schmidt Sciences call, with @aria-research.bsky.social, the @coop-ai.bsky.social (whose board I chair), and others. Open globally, due Aug 9.

A few details for anyone on the fence. We're aiming for a cohort of 20 to 25 fellows and we're reviewing applications as they come in. Earlier is better, but get yours in by July 15 to be assured of full consideration. bsky.app/profile/ghad...

Applications are open for a new AGI Governance Fellowship at Johns Hopkins, led by @sethlazar.org, Nicholas Caputo, and me. Three weeks in DC this September, in person, for early-career people already in AI governance. A capstone for the next generation, not an entry point.

My new op-ed is out in the Washington Examiner. The Great American AI Act would have the most powerful AI models tested for catastrophic risk by independent, government-licensed verifiers, with the government, not the companies, setting the standard. buff.ly/G45GK0Q

My colleagues at Fathom have a sharp new piece: independent verification has gone from a proposal to a live national debate. I've been working on this since 2017. This shift is real progress, but only the first step, and the harder question is what it should look like.

1/ Grateful for Dean Ball's sharp and balanced take on politics and AI governance, and the crazy situation we've landed in: increasingly urgent to get a smart regulatory framework for powerful AI, and yet basic rule of law and common sense are getting kicked to the curb. buff.ly/JfV0BiN

In a new essay, Anthropic CEO Dario Amodei calls for mandatory third-party testing of frontier AI models, with the government empowered to block unsafe deployments. One way to do it, in his words: a regulatory markets approach. His co-founder Jack Clark and I proposed that in 2019. buff.ly/rmhlHZn

New interview with Sinead Bovell on I’ve Got Questions. My version of existential risk for AI isn’t rogue superintelligence. It’s releasing billions of AI agents into the economy before we have legal infrastructure for identity, registration, or liability.

1/ The companies building AI are making consequential value choices every day. Right now, they are making those choices for the rest of us. New piece with @AndrewFATHOM on @Fathom_org Substack.

Thanks to Anton Korinek and UVA’s EconTAI Initiative for hosting me today to present work with Rakshit Trivedi and Dylan Hadfield-Menell on building AI agents that can participate in democratic life without destabilizing it. Paper at the Knight First Amendment Institute:

@deanwb.bsky.social in The Economist makes the public case for an architecture Dean and I have both been advancing: a network of independent organizations that audit AI safety claims. Worth a read. buff.ly/gZZCh8D

The second TAIGR workshop at ICML is open for submissions through April 24. Full papers and mini papers both welcome.If you're working on technical AI governance, send it in. If you know someone who is, pass this along. taigr-workshop.com

1/ Who decides how AI systems behave? With Tianmin Shu, @sethlazar.org, and @dhadfieldmenell.bsky.social, we were named runner-up in the LaudeInstitute's inaugural Moonshot program and awarded a seed grant to find out. www.laude.org/moonshots

1/ Something I’ve been working toward for a long time. Virginia just signed the first state legislation directing a formal study of Independent Verification Organizations for AI. Bipartisan votes of 84-14 in the House, unanimous 40-0 in the Senate.

1/ Social media makes it look like the public is deeply divided. But what if most of the public just isn’t speaking? In a new paper with Atrisha Sarkar in JAIR, we show why. We call it rational silence.

Democracy isn't a rulebook. It runs on daily interactions where people comply with norms and hold each other accountable. AI agents are about to join that system. We need to build them to read it. New paper with Rakshit Trivedi and Dylan Hadfield-Menell.

At the @FAR.AI London Alignment Workshop I made the case for Independent Verification Organizations: licensed, competing private entities that can grow our regulatory capacity to match the pace of AI capabilities. Full talk: bsky.app/profile/far....

AI systems are quickly becoming embedded throughout the economy. But we have almost none of the regulatory tools, regulatory markets among them, to manage them. Here's what I think we should do about it: www.americanbar.org/groups/scien...

“The most practical governance framework currently in circulation.” That’s Forbes on the Independent Verification Organization model Fathom and I have been developing. Legislation takes years; IVOs move at the pace of innovation.

"Why not work on what kind of new governance is needed to ensure secure, reliable, predictable use of all frontier models, from all companies?"

In London today and tomorrow for the Alignment Workshop organized by FAR.AI. Keynoting alongside Rohin Shah and Allan Dafoe. I look forward to seeing everyone in attendance! www.far.ai/events/event...

The 2026 AI Safety Report's biggest finding isn't the risks it catalogs. It's the evidence gap. We're trying to build AI governance with almost no science underneath. Massive investment in the research regulatory systems depend on is overdue. internationalaisafetyreport.org

Why not work on new governance...

NIST just launched an AI Agent Standards Initiative for identity, security, and interoperability. AI agents are becoming economic actors with zero legal infrastructure in place. We require businesses to register to operate. Why expect less of AI agents? buff.ly/kTU2cfX

In Paris this week for IASEAI (Feb 24-26). Tuesday: panel on the International AI Safety Report. Thursday: keynote on regulatory markets, a panel on AI assurance, and a talk in Seth Lazar’s workshop on normative competence. If you’re at IASEAI, come say hello!

Congratulations to Yoshua Bengio and the 39 other experts appointed to the UN’s first Independent International Scientific Panel on AI. 117-2 in the General Assembly.

New in Lawfare from Justin Curl, Sayash Kapoor, & Arvind Narayanan: AI won’t automatically make legal services cheaper. I’ve been working on this for a long time, legal markets are broken because of adversarial dynamics, credence goods problems, & regulations that protect incumbents, not consumers.

Billions going into building AI, barely any into making sure it works for us. Talked with @kevintfrazier.bsky.social & Andrew Freedman about our proposal making its way through state legislatures to build a competitive market for AI oversight. New @scalinglaws.bsky.social podcast:

1/ What makes self-interested AI agents cooperate? Not fine-tuning. Not central oversight. Gossip.

1/4 Moltbook now has 1.4 million AI agents—posting, voting, debating, running crypto scams. Humans can only observe. It's being called the "singularity." I'd call it a preview of the legal chaos I warned about in Fortune back in 2024. www.forbes.com/sites/guneyy...

What if "informal" institutions aren't so informal? Communities using elders for disputes are often called informal. We found key markers of legal formality—not in formal sources, but in people's beliefs and behavior. New paper on the Turkana: royalsocietypublishing.org/rstb/article...

Hiring a postdoc for the Normativity Lab at Johns Hopkins (2026 start). Looking for multiagent systems expertise (RL/generative agents) + interdisciplinary background in AI and cognitive science/econ/cultural evolution. apply.interfolio.com/177701

(1/2) 99% of surveyed businesses have lost money from AI failures—two-thirds lost over $1M, according to Ernst & Young. Insurance companies are stepping in: meet verifiable safety standards, get coverage. Don't meet them, you're on your own. I spoke with NBC News: buff.ly/wkmPooC

The recording of my keynote from #COLM2025 is now available!

I finally got a chance to meet @thomaslfriedman.bsky.social, whose book The World Is Flat inspired my own Rules for a Flat World. I had a great conversation with him and Andrew Freedman about the challenge we find the world facing: how do we build rules for AI that work in a complex world?

Human cooperation evolved through complex norms and institutions. Now we're introducing powerful new AI actors into our economic systems. At a recent workshop hosted at ASU we explored what evolution teaches us about getting the rules right.

Grateful to keynote at #COLM2025. Here's what we're missing about AI alignment: Humans don’t cooperate just by aggregating preferences, we build social processes and institutions to generate norms that make it safe to trade with strangers. AI needs to play by these same systems, not replace them.

Using debate among AI agents has been proposed as a promising strategy for improving AI reasoning capabilities. Our new research shows that this strategy can often have the opposite effect - and the implications for AI deployment are significant. (1/10) arxiv.org/abs/2509.05396

My lab @johnshopkins is recruiting research and communications professionals, and AI postdocs to advance our work ensuring that AI is safe and aligned to human well-being worldwide. 1/5

Six years ago @jackclarksf.bsky.social and I proposed regulatory markets as a new model for AI governance that would attract more investment---money and brains—in a democratically legitimate way, fostering AI innovation while ensuring these powerful technologies don’t 1/2

In this insightful interview, AIhub ambassador Kumar Kshitij Patel met @ghadfield.bsky.social‬, keynote speaker at ‪@ijcai.org, to find out more about her interdisciplinary research, career trajectory, AI alignment, and her thoughts on AI systems in general. aihub.org/2025/05/22/i...

Our latest monthly digest features: -Ananya Joshi on healthcare data monitoring -AI alignment with @ghadfield.bsky.social‬ -Onur Boyar on drug and material design -Object state classification with Filippos Gouidis aihub.org/2025/05/30/a...

Everyone, including those who think we're building powerful AI to improve lives for everyone, should take seriously how poorly our current economic indicators (unemployment, earnings, inflation) capture the well-being of low- and moderate-income folks. www.politico.com/news/magazin...

I was at this meeting Mon, and the quality & seriousness of discussion made it a highlight. But Fu Ying is right that forging the cooperation needed, even limited to the extreme risks that threaten everyone, is becoming ever harder. We must keep trying. www.scmp.com/news/china/d...

Do we think Musk is using treasury payments data to train, fine tune or do inference on AI models? @caseynewton.bsky.social

Video from our tutorial @NeurIPSConf 2024 is up! @dhadfieldmenell @jzl86 @rstriv and I explore how frameworks from economics, institutional and political theory, and biological and cultural evolution can advance approaches to AI alignment neurips.cc/virtual/2024...

“We haven’t created the infrastructure to integrate [agents] into all the rules and structures we have to make sure our markets behave well,” says CS faculty @ghadfield.bsky.social, an expert in #AI governance.

The most immediate need for #AISafety is more visibility by government into frontier models—we can’t determine if or how to regulate without it. The registration requirement Tino Cuéllar and Tim O’Reilly and I proposed last year is aimed at this goal. This white paper develops the proposal in depth