🚨New AI Safety Course @aims_oxford ! I’m thrilled to launch a new called AI Safety & Alignment (AISAA) course on the foundations & frontier research of making advanced AI systems safe and aligned at @UniofOxford what to expect 👇 robots.ox.ac.uk/~fazl/aisaa/

Evaluating the Infinite 🧵 My latest paper tries to solve a longstanding problem afflicting fields such as decision theory, economics, and ethics — the problem of infinities. Let me explain a bit about what causes the problem and how my solution avoids it. 1/N arxiv.org/abs/2509.19389

🚀 Excited to have 2 papers accepted at #NeurIP2025! 🎉 congrats to my amazing co-authors! More details (and more bragging) soon! and maybe even more news on sep 25 👀 See you all in… Mexico? San Diego? Copenhagen? Who knows! 🌍✈️

🚨 NEW PAPER 🚨: Embodied AI (incl. AI-powered drones, self-driving cars and robots) is here, but policies are lagging. We analyzed the EAI risks and found significant gaps in governance arxiv.org/pdf/2509.00117 Co-authors Jared Perlo @fbarez.bsky.social Alex Robey & @floridi.bsky.social 1\4

Other works have highlighted that CoTs ≠ explainability alphaxiv.org/abs/2025.02 (@fbarez.bsky.social), and that intermediate (CoT) tokens ≠ reasoning traces arxiv.org/abs/2504.09762 (@rao2z.bsky.social). Here, FUR offers a fine-grained test if LMs latently used information from CoTs for answers!

It is so easy to confuse chain of thought and explainability and in fact in a lot of the media it is presented as if with current LLMs we are allowed to view their actual thought processes. It is not that!

Excited to share our paper: "Chain-of-Thought Is Not Explainability"! We unpack a critical misconception in AI: models explaining their steps (CoT) aren't necessarily revealing their true reasoning. Spoiler: the transparency can be an illusion. (1/9) 🧵

Technology = power. AI is reshaping power — fast. Today’s AI doesn’t just assist decisions; it makes them. Governments use it for surveillance, prediction, and control — often with no oversight. Technical safeguards aren’t enough on their own — but they’re essential for AI to serve society.

And Anna Yelizarov, @fbarez.bsky.social, @scasper.bsky.social, Beatrice Erkers, among others. We'll draw from political theory, cooperative AI, economics, mechanism design, history, and hierarchical agency.

This is a step toward targeted, interpretable, and robust knowledge removal — at the parameter level. Joint work with Clara Suslik, Yihuai Hong, and @fbarez.bsky.social, advised by @megamor2.bsky.social 🔗 Paper: arxiv.org/abs/2505.22586 🔗 Code: github.com/yoavgur/PISCES

Come work with me at Oxford this summer! Paid research opportunity to: White-box LLMs & model security Safe RL & reward hacking Interpretability & governance tools Remote or Oxford. Apply by 30 May 23:59 UTC. DM with questions.

Come work with me at Oxford! We’re hiring a Postdoc in Causal Systems Modelling to: - Build causal & white-box models that make frontier AI safer and more transparent - Turn technical insights into safety cases, policy briefs, and governance tools ] DM if you have any questions.

First-time Area Chair seeking advice! What helped you most when evaluating papers beyond just averaging scores? After suffering through unhelpful reviews as an author, I want to do right by papers in my track.

🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉 > Follow @actinterp.bsky.social > Website actionable-interpretability.github.io @talhaklay.bsky.social @anja.re @mariusmosbach.bsky.social @sarah-nlp.bsky.social @iftenney.bsky.social Paper submission deadline: May 9th!

Organizers: Ben Bucknall, @lisasoder.bsky.social, @ankareuel.bsky.social @fbarez.bsky.social, @carlosmougan.bsky.social Weiwei Pan, Siddharth Swaroop, @ankareuel.bsky.social , Robert Trager @maosbot.bsky.social

Technical AI Governance (TAIG) at #ICML2025 this July in Vancouver! Credit to Ben and Lisa for all the work! We have a new centre at Oxford working on technical AI governance with Robert Trager and @maosbot.bsky.social many other great minds. We are hiring - please reach out! Quote

Life update: I'm starting as faculty at Boston University @bucds.bsky.social in 2026! BU has SCHEMES for LM interpretability & analysis, I couldn't be more pumped to join a burgeoning supergroup w/ @najoung.bsky.social @amuuueller.bsky.social. Looking for my first students, so apply and reach out!

New paper alert! Curious how small prompt tweaks impact LLM accuracy but don’t want to run endless inferences? We got you. Meet DOVE - a dataset built to uncover these sensitivities. Use DOVE for your analysis or contribute samples -we're growing and welcome you aboard!

What happens once AI can design better AI, which can itself design better AI? Will we get an "intelligence explosion" where AI capabilities increase very rapidly? Tom Davidson, Rose Hadshar and I have a new paper out with analysis of these dynamics.

My group @FLAIR_Ox is recruiting a postdoc and looking for someone who can get started by the end of April. Deadline to apply is in one week (!), 19th of March at noon, so please help spread the word: my.corehr.com/pls/uoxrecru...

1/13 LLM circuits tell us where the computation happens inside the model—but the computation varies by token position, a key detail often ignored! We propose a method to automatically find position-aware circuits, improving faithfulness while keeping circuits compact. 🧵👇

🔍 Excited to share our paper: "Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness"!

New paper alert! 🚨 Important question: Do SAEs generalise? We explore the answerability detection in LLMs by comparing SAE features vs. linear residual stream probes. Answer: probes outperform SAE features in-domain, out-of-domain generalization varies sharply between features and datasets. 🧵

🚨New arXiv preprint!🚨 LLMs can hallucinate - but did you know they can do so with high certainty even when they know the correct answer? 🤯 We find those hallucinations in our latest work with @itay-itzhak.bsky.social, @fbarez.bsky.social, @gabistanovsky.bsky.social and Yonatan Belinkov

We are excited to welcome Fazl Barez @fbarez.bsky.social, who joins us as a senior postdoctoral research fellow. He will be leading research initiatives in AI safety and interpretability. @oxmartinschool.bsky.social Find out more: www.oxfordmartin.ox.ac.uk/people/fazl-...

Very interesting paper about unlearning for AI Safety, a subject that deserves more attention. ⬇️

🚨 New Paper Alert: Open Problem in Machine Unlearning for AI Safety 🚨 Can AI truly "forget"? While unlearning promises data removal, controlling emergent capabilities is a inherent challenge. Here's why it matters: 👇 Paper: arxiv.org/pdf/2501.04952 1/8

What happens when "If at first you don't succeed, try again?" meets modern ML/AI insights about scaling up? You jailbreak every model on the market😱😱😱 Fire work led by @jplhughes.bsky.social Sara Price @aengusl.bsky.social Mrinank Sharma Ethan Perez arxiv.org/abs/2412.03556

🚨🛡️ Jailbreak Defense in a Narrow Domain 🛡️🚨 Jailbreaking is easy. Defending is hard. Might defending against a single, narrow behavior be easier? Even in this focused setting, all defenses fail 😱 arxiv.org/abs/2412.02159 Appearing at @AdvMLFrontiers (Oral) & @solarneurips #NeurIPS2024

Today is a good day for AI Safety! We are launching the AI Luminate AI Safety Benchmark @MLCommons @PeterMattson100 @tangenticAI The first step towards global standard benchmark for AI PRODUCT safety!

Our new Chancellor wants us admission tutors to "vet" Chinese applicants. How the heck are we supposed to do that? We're underpaid and overworked academics, not a national intelligence service www.politico.eu/article/oxfo...

ML /AI Safety researchers how do you find time to read papers, work on your projects and be so active on b sky, x etc?

🚨 Join us at NeurIPS 2024! 🚨 🧠 PrivacyML: Meaningful Privacy-Preserving ML & Evaluations 🌐 privacy.github.io Tackling the questions in AI Privacy & Safety: How do we protect training data privacy? What does unlearning mean for AI safety? Can cryptography make AI safer?