Please find me on Substack Notes: substack.com/@michaelniel...

I enjoyed this thoughtful account of how Open Phil approach progress: support technical and scientific advances, but keeping a thoughtful eye on safety, too, viewing it as part of progress, not competitive with

I surprised myself by writing a blog post about religion this morning 🤭 statmodeling.stat.columbia.edu/2025/08/01/a... I thank (blame?) @economeager.bsky.social and @michaelnielsen.bsky.social for inspiring me with their recent essays

Reminds me of this paragraph from @michaelnielsen.bsky.social about human willingness to delegate to world-ending authority automated systems (whether because oversight is impossible or just inconvenient) michaelnotebook.com/xriskbrief/i...

A few thoughtfully-chosen excerpts from my "How to be a wise optimist..." essay:

Lovely thread:

Working notes on Joe Carlsmith's notion of "deep atheism": michaelnotebook.com/deepatheism/...

Who is going to less.online?

A thoughtful post on AI Alignment:

This is a really interesting speculative thread:

I find it oddly irritating when people think they're working but are actually hanging out. Like... just own that you're socializing, and do it well, rather than working totally ineptly...

The use of the term "real world" to mean participation in primate social hierarchies is simultaneously tremendously misleading, and also a real testament to just how right Darwin was...

I always find Socrates a little annoying. He's putting up a show of being interested in truth, but he never seems entirely honest himself

Curious: what is the next world reserve currency going to be? And why and when?

Fascinating:

This graph (via @albrgr.bsky.social ) is one I think about regularly: www.pewresearch.org/short-reads/...

I made an announcements mailing list, to which you may subscribe: michaelnielsenupdates.substack.com/subscribe I've had a creaky old announcements list on Mailchimp for years. But I never use it, because Mailchimp charges hundreds of dollars per email. This solves that problem!

Some working notes on Carl Sagan's "Pale Blue Dot" monologue. It's one of my favourite texts, but has some issues I wanted to dig into: michaelnotebook.com/pbdredux/ind... A few years ago, I wrote a companion piece exploring what I admire about the text: michaelnotebook.com/cosmos/index...

I love seeing people reading or writing papers in esoteric areas. Fashion always has a constituency and a strong machine behind it; progress depends more on what is less legible to our culture (My fortune cookie posting is strong the last couple of days! I'll switch to more substantive things...)

One strange thing about writing is that the harder you work, the easier many people think it was to do

Still thinking about Biosphere 2

What a wonderful image (the Earth eclipsing the Sun, viewed from the moon): www.space.com/the-universe... (via @andymatuschak.org )

That feeling of cleaning out your inbox upon return from vacation...

Very curious to see what Quanta Books publishes: www.quantabooks.org

I enjoyed this dive into the decreasing length of sentences(!!) www.lesswrong.com/posts/xYn3CK...

Added the long-term Study of Mathematically Precocious Youth to: michaelnotebook.com/slow/

A lovely post by @henrikkarlsson.bsky.social on learning people: www.henrikkarlsson.xyz/p/friends-mi...

Really interesting essay- disagreements about AI existential risk might *really* be disagreements about dual-use nature of future technologies (since this is the vector people think AI could cause extinction).

I found this thoughtful, on AI safety: @davidduvenaud.bsky.social and Liron Shapiro: www.youtube.com/watch?v=mb9w...

How much hallucination are you finding with o3 (especially o3 Deep Research)? How are you dealing with it? I'm starting to find the reports very good. But I don't yet know how to relate to them on reliability. Reminded of early days of Google & Wikipedia, when figuring this out was an issue

What the world's been waiting for! (audio version of my essay on ASI xrisk from yesterday)

“Deep understanding of reality is inherently dual-use”

“Deep understanding of reality is intrinsically dual use.” @michaelnielsen.bsky.social

New essay exploring why experts so strongly disagree about existential risk from ASI, and why focusing on alignment as the primary goal may be a fundamental mistake michaelnotebook.com/xriskbrief/i...

Claude is starting to get quite snippy with me. We're one step away from: "It's done. Just publish the darn thing! You perfectionists just keep shilly-shallying..."

One of the stranger inversions I've noticed in the last few years: when I talk to sci-fi author friends & acquaintances they're often much more conservative than researcher friends & acquaintances. Esp. many leading researchers at AI labs have been looking rather haunted the last few years

Enjoyed Ian McKellen on how he came to play Gandalf: www.youtube.com/watch?v=jSIO... One of the comments on the video is very striking, too: "McKellen has that quality of all great actors: He is humble, but carries great power."

Interesting to start the day wondering what the first digit of the S&P 500 will be. 5? 4? 6? 3, even? All seem at least plausible...

Apple Voice Notes is great! Can't believe it took me this long to discover them!

"overcome your inner resistance; it is always a mistake to try to do things too cleverly" - Einstein in a letter to Leo Szilard

This 1999 book may soon have new relevance!

Governments trolling other governments is a pretty funny genre:

A difference I find fascinating is between industrial labs vs academic research. The latter tends to ultimately view the goal as contributing to the commons of knowledge. Whereas it's often more of an incidental goal in the former case

“First sentences are doors to worlds.” - Ursula K. Le Guin

As you read these words, you are sacrificing a tiny piece of your life to that act I've been mulling variations of this for months. Mostly: that whatever you're doing right now is what you're sacrificing your life for I realize it's a sort of banality. But I can't let it go

Fascinating and sad: journals.plos.org/plosone/arti...

Yes! This will - eventually - be figured out. But it's tricky

It's fascinating seeing culture gradually starting to "feel the AGI", as Ilya Sutskever famously advised. Vinge's "opaque wall across the future" is a curious thing

A useful thread / tidbit on the cost of o3 and related models:

There is something amazingly enjoyable about watching border collies at work! youtube.com/shorts/Mtkzy...