New AI governance research agenda from MIRI’s TechGov Team. We lay out our view of the strategic landscape and actionable research questions that, if answered, would provide important insight on how to reduce catastrophic and extinction risks from AI. 🧵1/10 techgov.intelligence.org/research/ai-...

MIRI's (@intelligence.org) Technical Governance Team submitted a comment on the AI Action Plan. Great work by David Abecassis, @pbarnett.bsky.social, and @aaronscher.bsky.social Check it out here: techgov.intelligence.org/research/res...

One mechanism that seems promising is Flexible Hardware-Enabled Guarantees (FlexHEGs) and on-chip approaches. These could potentially be used to securely carry out a wide range of governance operations on AI chips, without leaking sensitive information. (1/3)

Reflection: The more I got into the weeds on this project, the harder verification seemed. Some difficulties are distributed training, algorithmic progress, and the need to be robust against state-level adversaries. It’s hard, but we have to do it! (1/1)

One mechanism that seems promising: Signatures of High-Level Chip Measures. Classify workloads (e.g., is it training or inference) based on high-level chip measures like power-draw, but using ‘signatures’ of these measures based on temporary code access. (1/6)

One mechanism that seems especially promising: Networking Equipment Interconnect Limits, like “Fixed Sets” discussed by www.rand.org/pubs/working... but can be implemented with custom networking equipment quickly. (1/8)

One thread throughout this report is that low-tech, high-access solutions can often substitute for high-tech, low-access solutions, let’s walk through some examples. (1/12)

Distributed training (i.e., geographically distributed, decentralized) could pose major problems for many AI Governance plans. In the default case, large AI training happens in a small number of big data centers, so monitoring training can focus on those data centers. (1/10)

Inspectors who have full access to your systems seem like they could pose a major privacy and security risk, so it may be necessary to have very tight info sec around them, e.g., limited communication to home countries. (1/3)

Reflection: People I talked to had wildly different intuitions about the likelihood of direct US/China conflict in response to the threat of US AGI and ASI (superintelligence) development. (1/4)

How can the US and China (or the international community, broadly) ensure compliance in AI agreements to manage large-scale risks? In a recent report, we discuss options available for verification of international AI treaties. Applicable to domestic rules too! 🧵 (1/12)

Alright here's a thread with some takes on recent AI safety-ish papers, written as I skim them because I want more content on this platform