No one has a plan for human flourishing after AGI, nor a clear understanding of what’s even possible. So we’re hosting a 2-day workshop at LightHaven on these topics! www.post-agi.org 🧵

Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below! with @alecrad.bsky.social and @nicklevin01.bsky.social

my coworkers at ACS published a new paper: What determines AIs’ self-conception? theartificialself.ai Because AIs can be copied, rewound, and edited, they have different options for selfhood than humans. This is still malleable, and influences important behaviors such as self-preservation. 🧵

A new paper co-authored by SRI Chair @davidduvenaud.bsky.social examines “gradual disempowerment”: how incremental AI deployment could steadily reduce human influence over the economy, culture, and the state—without a single abrupt takeover. 80000Hours feature: 80000hours.org/podcast/epis...

My interview with Rob Wilblin on Gradual Disempowerment is up: www.youtube.com/watch?v=XV3e... I make the case that, even if we solve the technical problem of aligning powerful AIs, that our institutions, culture, and governments will serve us less well once we're all drags on growth.

How might the world look after the development of AGI, and what should we do about it now? Help us think about this at our workshop on Post-AGI Economics, Culture and Governance! We’ll host speakers from political theory, economics, mechanism design, history, and hierarchical agency. post-agi.org

Me and Raymond Douglas on how AI job loss could hurt democracy. “No taxation without representation” summarizes that historically, democratic rights flow from economic power. But this might work in reverse once we’re all on UBI: No representation without taxation! bsky.app/profile/econ...

It's hard to plan for AGI without knowing what outcomes are even possible, let alone good. So we’re hosting a workshop! Post-AGI Civilizational Equilibria: Are there any good ones? Vancouver, July 14th www.post-agi.org Featuring: Joe Carlsmith, @richardngo.bsky.social‬, Emmett Shear ... 🧵

What to do about gradual disempowerment from AGI? We laid out a research agenda with all the concrete and feasible research projects we can think of: 🧵 www.lesswrong.com/posts/GAv4DR... with Raymond Douglas, @kulveit.bsky.social @davidskrueger.bsky.social

On top of the AISI-wide research agenda yesterday, we have more on the research agenda for the AISI Alignment Team specifically. See Benjamin's thread and full post for details; here I'll focus on why we should not give up on directly solving alignment, even though it is hard. 🧵

“What place will humans have when AI can do everything we do — only better?” In The Guardian today, SRI Chair @davidduvenaud.bsky.social explores what happens when AI doesn't destroy us — it just quietly replaces us. 🔗 www.theguardian.com/books/2025/m... #AI #AIEthics #TechAndSociety

My single rule for productive Bluesky discussions: Start every single reply with a point of agreement. It disarms the combative impulse on both sides, and forces you to try to interpret their words in the most sensible possible way.

New paper: What happens once AIs make humans obsolete? Even without AIs seeking power, we argue that competitive pressures are set to fully erode human influence and values. www.gradual-disempowerment.ai with @kulveit.bsky.social, Raymond Douglas, Nora Ammann, Deger Turann, David Krueger 🧵

Happy to have helped a little with this paper: