Today I’m publishing a series of research notes on the idea of an international AGI project. The aim is to assess how desirable an international AGI project is, and what the best version of such a project is (taking feasibility into account).

Beyond Existential Risk: In a new paper, Guive Assadi and I argue against Bostrom's "Maxipok" principle—that altruists should seek to maximize the probability of an "OK outcome," where OK just means avoiding existential catastrophe.

Almost no one has articulated a positive vision for what comes after superintelligence. What should we be trying to aim for? Utopias from history look clearly dystopian to us, and we should expect the same for our own attempts. We don’t know enough, or have the authority, to decide the details.

Merry Christmas, everyone! This year, I’m feeling grateful to be me. Recently, I gave some information about myself to Claude, and asked how I compared to other 38-yr-old men in the world.

What projects today could most improve a post-AGI world? In “How to make the future better”, I lay out some areas I see as high-priority, beyond reducing risks from AI takeover and engineered pandemics.

The trajectory of the future could soon get set in stone. In a new paper, I look at mechanisms through which the longterm future's course could get determined within our lifetimes.

Should we expect widespread moral progress in the future? In a new paper, Convergence and Compromise, @FinMoorhouse and I discuss this. Thread.

How easy is it to get to a future that’s pretty close to ideal? In “No Easy Eutopia”, @finmoorhouse and I argue that it’s very hard - very unlikely to happen by default, without some beings in the future trying hard to figure out what “goodness” consists in, and actively trying to promote it.

Today I’m releasing an essay series called Better Futures. It’s been something like eight years in the making, so I’m pretty happy it’s finally out! It asks: when looking to the future, should we focus on surviving, or on flourishing?

.@MTabarrok reviews my 80k podcast, pointing out that growth rates have already accelerated greatly in history, and yet the world is better than ever. This misunderstands both the article and the podcast.

What happens once AI can design better AI, which can itself design better AI? Will we get an "intelligence explosion" where AI capabilities increase very rapidly? Tom Davidson, Rose Hadshar and I have a new paper out with analysis of these dynamics.

How will the creation of AGI be governed? CERN and the Manhattan project are often suggested as models for an international AGI project. We think an under-discussed international organisation, Intelsat, provides a promising complement to these existing models. See 🧵1/

Is AGI an “all or nothing” problem? Failure on alignment = AI takeover, and success = AI solves everything? In a new paper with Fin Moorhouse, we argue no. We describe the dizzying range of challenges AGI will pose, *even if* we succeed at alignment. forethought.org/research/pre...

It’s been a real joy working with such a great team to set up a new research institute: Forethought. www.forethought.org Expect a lot of work soon! - on intelligence explosion dynamics, AGI governance, differential AI development, AI-assisted coups, & other challenges from rapid AI-driven change

I have a new book out! "An Introduction to Utilitarianism", co-authored with Richard Chappell and Darius Meissner. It all started in 2016: I was giving the introduction to ethics lectures at Oxford and I was frustrated by how utilitarianism was presented to students in introductory materials...

This week I donated to Malaria Consortium’s Seasonal Malaria Chemoprevention Program - one of @GiveWell's top-rated charities. If you donate $5000 (£4000) to this program, on average one fewer child will die from malaria.

I’ve just donated to @OurWorldInData They’re like the Office of National Statistics but for the whole world. By providing exceptionally clear information about the most important issues, they help decision-makers across the world make better-informed choices. ourworldindata.org/donate

Happy Birthday to Giving What We Can, which turns 15 today! 🥳 When Toby and I started GWWC 15 years ago, we had just 23 members, each of us pledging at least 10% of our income to effective charities until we retire. Now, over 9,000 people have taken this pledge. www.givingwhatwecan.org