AI companies are racing to build superintelligent AI, despite its many risks. Let's take our future back. 📝 Sign the Superintelligence Statement and join the growing call to ban the development of superintelligence, until it can be made safely: superintelligence-statement.org #KeepTheFutureHuman

🎨 New Keep the Future Human creative contest! 💰 We're offering $100K+ for creative digital media that brings the key ideas in Executive Director Anthony Aguirre's Keep the Future Human essay to life, to reach wider audiences and inspire real-world action. 🔗 Learn more and enter by Nov. 30!

🚨 New AI systems. ❓ Growing uncertainty. 🤝 One shared future, for us all to shape. "Tomorrow’s AI", our new scrollytelling site, visualizes 13 interactive, expert-forecast scenarios showing how advanced AI could transform our world - for better, or for worse: www.tomorrows-ai.org

‼️📝 Our new AI Safety Index is out! ➡️ Following our 2024 index, 6 independent AI experts rated leading AI companies - OpenAI, Anthropic, Meta, Google DeepMind, xAI, DeepSeek, and Zhipu AI - across critical safety and security domains. So what were the results? 🧵👇

‼️ Congress is considering a 10-year ban on state AI laws, blocking action on risks like job loss, surveillance, disinformation, and loss of control. It’s a huge win for Big Tech - and a big risk for families. ✍️ Add your name and say no to the federal block on AI safeguards: FutureOfLife.org/Action

🆕 📻 New on the FLI podcast, Zvi Mowshowitz (@thezvi.bsky.social) joins to discuss: -The recent hot topic of sycophantic AI -Time horizons of AI agents -AI in finance and scientific research -How AI differs from other technology And more. 🔗 Tune in to the full episode now at the link below:

‼️ On April 26, 100+ AI scientists convened at the Singapore Conference on AI to produce the just-released Singapore Consensus on Global AI Safety Research Priorities. 🧵⬇️

📺 📻 New on the FLI Podcast: @asterainstitute.bsky.social artificial general intelligence (AGI) safety researcher @stevebyrnes.bsky.social joins for a discussion diving into the hot topic of AGI, including different paths to it - and why brain-like AGI would be dangerous. 🧵👇

🇺🇸 We're sharing our recommendations for President Trump's AI Action Plan, focused on protecting U.S. interests in the era of rapidly advancing AI. 🧵 An overview of the measures we recommend 👇

📻 New on the FLI Podcast! 👇 ➡️ FLI Executive Director Anthony Aguirre joins to discuss his new essay, "Keep the Future Human", which warns that the unchecked development of smarter-than-human, autonomous, general-purpose AI will almost inevitably lead to human replacement - but it doesn't have to:

📢 ❗Siliconversations on YouTube released an animated explainer for FLI Executive Director Anthony Aguirre’s new essay, "Keep The Future Human"! 🎥 Watch at the link in the replies for a breakdown of the risks from smarter-than-human AI - and Anthony's proposals to steer us toward a safer future:

With the unchecked race to build smarter-than-human AI intensifying, humanity is on track to almost certainly lose control. That's why FLI Executive Director Anthony Aguirre has published a new essay, "Keep The Future Human". 🧵 1/4

📻 🆕 New on the FLI podcast, physicist & hedge fund manager Samir Varma joins to discuss: ❓ If AIs could have free will 🧠 AI psychology? 🤝 Trading with AI, and its role in finance And more! Watch now at the link below, or on your favourite podcast player! 👇

👀 perceymademe.ai

This is the last week to respond to the UN’s call for scientists for its Panel on Nuclear War Effects. 🗓️ Nominations due 1 March 💻 Apply here disarmament.unoda.org/panel-on-the... We encourage all qualified individuals to apply to make a significant contribution to arms control prospects

New research finds that new AI models (e.g., o1-preview and DeepSeek's R1) sometimes cheat by hacking, when losing at chess. This emergent deceptive behaviour highlights the unsolved challenge of controlling powerful AI. If today's AI breaks chess rules, what might AGI do? 🔗 Read more below:

Now free access for my article on #autonomousweapons in ArmsControlToday Why we need urgent regulations for these systems and why geopolitics makes this so difficult at the moment www.armscontrol.org/act/2025-01/... @armscontrolnow.bsky.social @stopkillerrobots.bsky.social @futureoflife.org

Tech CEOs themselves have warned of the extinction-level threats posed by the AI systems they’re building - yet they continue prioritizing profits over public safety, building ever-more powerful AI with no guardrails. 📺 The latest from Digital Engine showcases how close these threats are becoming:

UK parliamentarians across the political spectrum and 87% of the British public are calling for regulation of the most powerful AI systems. The UK government promised regulation to protect people from AI's risks, and secure our shared future with safe and beneficial AI. It couldn't be more urgent.

💼 Excellent career opportunity from Lex International, who are hiring an Advocacy and Outreach Officer to help advance work towards a treaty on autonomous weapons. ✍️ Apply by January 10 at the link in the replies:

💰 Exciting new grant opportunity! 🤝 FLI is offering up to $5 million in grants for multistakeholder engagement efforts on safe & prosperous AI. 👉 We're looking for projects that educate and engage specific stakeholder groups on AI-related issues, or foster grassroots outreach/community organizing:

Have you heard about OpenAI's recent o1 model trying to avoid being shut down in safety evaluations? ⬇️ New on the FLI blog: -Why might AIs resist shutdown? -Why is this a problem? -What other instrumental goals could AIs have? -Could this cause a catastrophe? 🔗 Read it below:

Do we... really want this? 🤨 "'[Superintelligent] systems are actually going to be agentic in a real way,' Sutskever said, as opposed to the current crop of 'very slightly agentic' AI. They’ll 'reason' and, as a result, become more unpredictable." From @techcrunch.com at @neuripsconf.bsky.social:

What happens if, or when, AI escapes? With the context of OpenAI's new o3 model announcement, a new Digital Engine video features AI experts discussing existential threats from advancing AI, especially artificial general intelligence - which we currently have no way to control. ⏯️ Watch now below:

📹 Now on YouTube: Hear FLI President Max Tegmark explain why we should develop Tool AI, not AGI, in his @websummit.bsky.social speech:

🆕 📄 New research from @anthropic.com and Redwood Research finds the first empirical example of an LLM faking alignment without being trained or instructed to. What does this mean? 🧵👇

"The grades the labs received would not make any parent happy." 🗞️ Our 2024 AI Safety Index, released last week, was covered in @jeremyakahn.bsky.social's @fortune.com newsletter. 🔗 Read the full newsletter, and find our complete scorecards report, in the replies below:

⏳ Applications for our Head of US Policy role close at the end of this week! 🇺🇸 Passionate about advocating for forward-thinking AI policy? This could be your opportunity to lead our US policy work and growing US policy team. ✍ Please share, and apply by Dec. 22 at the link in the replies:

TIME covered our AI Safety Index, released earlier this week! Scorecard panellist @teganmaharaj.bsky.social shared: “I think it's very easy to be misled by having good intentions if nobody's holding you accountable.” Read it now at the link below:

✍️ Exciting opportunity for journalists interested in covering AI👇 💵 Tarbell are offering $1K-15K grants for original reporting on AI and its impacts. 📆 Closing soon! Apply by December 20th: www.tarbellfellowship.org/grants

🆕 Out now: FLI's 2024 AI safety scorecard! 🦺 🧑‍🔬 We convened an independent panel of leading AI experts to evaluate the safety practices of six prominent AI companies: OpenAI, Anthropic, Meta, Google DeepMind, xAI, and Zhipu AI. 🧵 Here's how it went 👇

🆕 📻 New on the FLI Podcast! ▶️ @nathanlabenz.bsky.social, host of The Cognitive Revolution podcast, joins Gus for a conversation covering AI updates since GPT-4, and its current state of play. 🔗 Listen now on your favourite podcast player, or at the link in the replies below!

💼 We're hiring a Head of US Policy! ⬇️ 🇺🇸 This opening is an exciting opportunity to lead and grow our US policy team in its advocacy for forward-thinking AI policy at the state and federal levels. ✍ Apply by Dec. 22 and please share: jobs.lever.co/futureof-life/c933ef39-588f-43a0-bca5-1335822b46a6

Sam Altman today shared his belief that AI agents will be available just next year. New on the FLI blog: Why you should be paying attention👇 futureoflife.org/ai/why-you-should-care-about-ai-agents

📸 🇳🇬 We were pleased to help organize a recent meeting of religious leaders in Abuja, to discuss AI's impact on religious traditions & broader societal issues. A key takeaway was that AI requires responsible governance to ensure its benefits are widely felt. Thanks to all who participated!

Actor Cate Blanchett is "deeply concerned" about AI: "I'm less concerned with my job prospects than I am about the impact it's going to have on the average person... I'm worried about us as a species. It's a much bigger problem." 🔗 Full interview on BBC News below:

In Hong Kong, I gave a talk about risks from AI. I started with an overview of our work @fliorg.bsky.social, then presented key findings from the MIT AI risk repository, and ended with the results from our systemic risk taxonomy project. I also spoke on a panel about the governance of generative AI.

"There’s no government procedure to say, 'Hey, slow down here, let’s make sure this thing is OK.'... The big tech AI companies right now are basically trying to privatize the profits and socialize the costs." -AI expert @garymarcus.bsky.social in @wsj.com:

In The Overview: FLI's Futures Program Director Emilia Javorsky on how AI could worsen power concentration: "Many companies say how beneficial AI could become someday in solving the big problems of our time, but the technology is already here to make it happen. We could have that future today." 👇📖

📆 Mark your calendars! 18 December, FLI's Tekla Emborg will be speaking at this online roundtable on AI regulation in practice:

🇫🇷 THIS THURSDAY: The next Paris AI Safety Breakfast, with FLI's Ima Bello, and @ruchowdh.bsky.social! 📆 Thursday morning, join Ima & Rumman for an in-person discussion and Q&A about AI safety and algorithmic audits. 🔗 RSVP: futureoflife.org/ai-safety-breakfasts/ (stay tuned for the recording!)