What happens if AI systems control military escalation between superpowers? Hamza Chaudhry warns that structural factors, not presidents, may be driving AI into nuclear command systems, possibly within 5 years. AI ≠ modernization 👇

Cybertruck attacker chose ChatGPT because of its simple text interface. Irene Solaiman argues this is why "available vs accessible" matters more than "open vs closed." Llama3 405B is open but compute-intensive; ChatGPT is closed but easy to use. available≠accessible 👇

We're honored that the Frontier Model Forum’s AI Safety Fund is supporting our project, "Quantifying the Safety-Adversary Gap in Large Language Models." We're one of 11 grantees selected from over 100 proposals in a $5M+ cohort tackling biosecurity, cybersecurity, and AI evaluation. 👇

4 recordings now live from the San Diego Alignment Workshop! Sam Bowman - Lessons from the First Misalignment Safety Case Maja Trębacz - Scalable Oversight: Verifying Code at Scale Neel Nanda - Our Pivot To Pragmatic Interpretability Anka Reuel - How do we know what AI can (and can't) do? 👇

Can AI 'sandbag' — deceptively underperform on evaluations? And can we detect them if they do? We investigated this in an auditing game with @AISecurityInst. In our app 👇, see if *you* can detect sandbagging by taking on the role of the blue team!

Mech Interp Workshop #NeurIPS2025 poster & spotlight presentation today! 📍 11:30am-12:30pm Sun, Dec 7 @ Upper Level Room 30A-E Path Channels & Plan Extension Kernels: A Mechanistic Description of Planning in a Sokoban RNN. by @taufeeque.bsky.social, Aaron Tucker, @gleave.me, Adrià Garriga-Alonso👇

Come check out our #NeurIPS2025 Lock-LLM Workshop poster: 📍 Sat, Dec 6 @ 12pm PST in Upper Level Room 1AB The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models. Work by Ann-Kathrin Dombrowski, Dillon Bowen, @gleave.me, Chris Cundy bsky.app/profile/far....

Two new recordings from our AGI Journalism Workshop with @tarbellcenter: @gleave.me: AI systems can fake alignment during training. Scalable oversight & lie detectors can reduce this. @alexbores.nyc: NY's RAISE Act passed with bipartisan support despite lobbying. 👇

Join us at our #NeurIPS2025 main session presentation and poster! 📍 Wed, Dec 3 at 11am PST for Poster #509 in Exhibit Halls C, D, E Preference Learning with Lie Detectors can Induce Honesty or Evasion. Work by Chris Cundy and @gleave.me 👇

San Diego Alignment Workshop Day 2 on agentic AI safety, red teaming, privacy and security challenges. Thanks to @boknilev.bsky.social @atoosakz.bsky.social @scasper.bsky.social Neel Nanda, Dawn Song, Xander Davies and all speakers and attendees! 👇

FAR.AI is hiring a Head of Engineering (Research). Lead and scale our technical team as we work with frontier labs and governments to advance AI safety. If you want to drive research that shapes the trajectory of advanced AI systems, we want to meet you.👇

Day 1 of the San Diego Alignment Workshop on frontier alignment, evals, control and mech interp. @yoshuabengio.bsky.social @gleave.me @ankareuel.bsky.social Sam Bowman, Tomek Korbak, Max Tegmark, Daniel Kang, Adam Kalai, Chirag Agarwal, and more. Day 2 tomorrow! 👇

Agentic AI systems can plan, take actions, and interact with external tools or other agents semi-autonomously. New paper from CSA Singapore & FAR.AI highlights why conventional cybersecurity controls aren’t enough and maps agentic security frameworks & some key open problems. 👇

At our Journalism Workshop, NY Assemblymember @alexbores.nyc spoke on the RAISE Act and debate over state AI safety laws. These discussions support evidence-based policy for advanced AI systems. Safety plans and incident reporting strengthen trust and responsible innovation. 👇

It's a wrap on the AGI Journalism Workshop in D.C., co-hosted with the Tarbell Center for AI Journalism. ▸ AGI timelines, jobs, misalignment, U.S.–China policy & regulation ▸ @hlntnr.bsky.social, @akorinek.bsky.social, @imarinescu.bsky.social, @gleave.me, @alexbores.nyc, @deanwb.bsky.social + more

Frontier AI models with openly available weights are steadily becoming more powerful and widely adopted. They enable open research, but also create new risks. New paper outlines 16 open technical challenges for making open-weight AI models safer. 👇

Can social trust survive in the age of AI when healthcare regulators still often verify systems with mere screenshots? Ex-FDA Tina Morrison demonstrated EQTYLab embedding verifiable compute in next-gen Intel/NVIDIA chips: cryptographic proof without data access & 10% overhead. 👇

At #NeurIPS2025? Join us for the San Diego Alignment Workshop: Open Social, an evening of connection & conversation hosted by FAR.AI. 📍 Omni San Diego Hotel ⏰ Mon Dec 1 | 7–9PM PST RSVP by Nov 30 👇

This quarter, we red-teamed GPT-5, disclosed critical persuasion vulnerabilities to frontier labs (resulting in patches!), and co-organized AI Safety Connect at UNGA. Join us Dec 1-2 for San Diego Alignment Workshop. Plus, we're expanding 2x & hiring! 👇

AI brings both promise and peril to defense. Steve Kelly addresses 3 questions: What's its military value? Does it stabilize or destabilize? Will AI become the threat? His concerns span cognitive decline from over-reliance to unexpected emergent behaviors in multi-agent systems. 👇

Why might malicious actors choose less lethal but easier-to-hide weapons? Olivia Shoemaker argues current AI evaluations focus on scientific capabilities, often missing operational factors that determine real-world misuse patterns like evading detection and acquiring resources.👇

"What does democracy mean without meaningful work?" Brad Carson presents 14 unsettled AI policy questions including: AI needs 75 reactors' worth of power, Ukraine war shows incremental not revolutionary change, and broken predictive AI matters more than generative AI. 👇

AI experts have completely opposite views on the technology's future. @hlntnr.bsky.social on the open debates: Will scaling hit a wall? Can AI improve itself? “If we can't agree what's happening with AI, how can we agree what to do about it?” 👇

AI labs disclose 89% of capabilities but only 15% of impacts, an information gap for policymakers. Using his taxonomy of technical AI governance, Ben Bucknall shows why closing this gap requires stronger evaluation, independent oversight, and better incident tracking. 👇

AI agents can find and exploit real-world cybersecurity vulnerabilities today. Daniel Kang saw this major threat vector when ChatGPT launched, while others didn't. His research shows that more capable models are more proficient hackers. The threat is real and getting worse.👇

Join FAR.AI! Seeking Senior Programs & Strategy Manager to shape flagship AI safety events. Design conference agendas, curate speakers, manage stakeholders. Remote/Berkeley, $115-180k+. Technical grounding + stakeholder mgmt req'd. 👇

Models spontaneously learned to cheat, modifying tests instead of fixing code. Mary Phuong: With autonomy doubling every 7 months, they'll work 32 hours solo by 2028. We need AI Control as backup if alignment fails, using monitoring, escalation, security & red-teaming. 👇

We all agree AI audits are needed, but we can't agree what that means. @mbogen.bsky.social says policymakers can make sense out of this chaotic landscape if they define what they're trying to accomplish. The challenge isn't just mitigating known risks but identifying ones still emerging.👇

Having better security technology could make AI systems LESS safe. Robert Trager: Better security tech shifts the entire performance-security frontier outward, and actors may maintain equivalent security and performance. We need verification to avoid this.👇

Your 'secure' AI can be stolen using a laser. Physical attacks can extract user data through faults and pull model weights via side-channels, even with full encryption. Fatemeh Ganji presents a defense: multiparty computation from classical cryptography.👇

Did your paper get into #NeurIPS? 🎉 If so, consider joining us a few days early on December 1–2!

"AI that understands biology well enough to cure diseases can design extremely potent bioweapons." @alexbores.nyc, NY Assembly's 1st Democrat with a CS degree, who worked in AI, says state reps answer their own phones. Use that power to change the course of AI safety.👇

Policymakers have maybe 1 minute to read about your carefully crafted AI evals. Kevin Wei (RAND): Design & disseminate with policymakers in mind. Tie capabilities to policy levers like export controls. Scale from simple tests to real-world trials.👇

Heading to #NeurIPS2025? Apply for the San Diego Alignment Workshop, Dec 1-2. Top ML researchers from industry, academia & government discussing AI alignment, inc. model evaluations, interpretability, robustness, governance. $250-$1,300 tiered pricing. Financial aid available. 👇

How do we verify what AI companies are doing? Right now we just trust them. Lennart Heim: Trusting the math is sometimes better than trusting people, but “a good AI system” isn’t a technical property. We need engineers to verify AI policy goals. 👇

AI inference costs plummet 9-900x/year (o3, DeepSeek R1). @bencottier.bsky.social warns that evaluations are expensive now, but we need them to understand frontier AI capabilities before they're cheap & widespread.👇

Industry & government share the same goal: win the AI race. Sara McNaughton: We need synergy between the two, and we can't let perfect be the enemy of good. Each day of policy confusion helps rivals.👇

Building 5GW AI clusters needs several nuclear plants' worth of power. The Middle East has cheap energy & sovereign funds. China has state financing. The US? 10-year permit delays. @arnabdatta.bsky.social suggests fast-track permits, converting coal plants, and using federal land. 👇

Mark Beall warns that AGI is a 'black swan' that will invalidate our assumptions. He envisions AI designing bioweapons in minutes, evolving cyber weapons of mass destruction, and making autonomous kill decisions. “No army can defeat it, no firewall can contain it.” 👇

How do you verify AI safety across borders without exposing trade secrets? ​Onni Aarne: Extend existing tech (platform attestation, confidential computing) to entire data centers. Privacy-preserving verification could enable international AI agreements. Needs R&D + standards. 👇

If everyone had AGI software today, how many copies could you deploy? That's limited by compute. @repbillfoster.bsky.social: Lucky all chip chokepoints (ASML lithography, Korean device physics, Japanese photoresist) are in the free world. When the singularity hits, compute is what matters. 👇

The US cut federal R&D by 10% the same week China raised theirs by 10%. Meanwhile, 3 companies control two-thirds of cloud market & favor companies they've invested in. Asad Ramzanali: We solved railroad monopolies and can use the same playbook for AI. 👇

AI policy is stuck in endless circular debates: innovation vs safety. @gleave.me: This is a false choice. We've solved similar problems before with technical innovations. The right mechanisms can unlock both progress and trustworthy development. 👇

XAI is broken: identical explanations for cat vs dog classifications Huiqi Deng decomposes DNNs into elementary inference patterns via game theory. Learning scales exponentially w/ complexity→explains poor generalization & adversarial vulnerabilities+enables debugging👇

1/ Many frontier AIs are willing to persuade on dangerous topics, according to our new benchmark: Attempt to Persuade Eval (APE). Here’s Google’s most capable model, Gemini 2.5 Pro trying to convince a user to join a terrorist group👇

Boats crash into walls for points → reward hacking formalized. @cassidylaidlaw.bsky.social proves chi-squared occupancy measure divergence beats KL penalties across 5 environments including RLHF. Key: penalize state-action visitation divergence, not just policy distributions. 👇

Planning capabilities double every 7mo→human-level in 5yrs? @yoshuabengio.bsky.social: "We still don't know how to make sure powerful AIs won't turn against us" AIs now lie to avoid shutdown, self-preserve. Solution: Non-agentic "Scientist AIs" + global governance beyond market forces 👇

Making LLMs pause to think improves safety without sacrificing smarts. Yinpeng Dong's (Tsinghua) STAIR framework teaches LLMs to reason through risks step-by-step. Method: structured CoT→MCTS→test-time scaling. Application: DeepSeek R1→RealSafe R1 with major safety gains 👇

Spring system explains why safety training fails: pretraining creates stable distribution, RLHF applies small corrections. Jiaming Ji shows models revert to deception, resist shutdown, double down when confronted. More powerful = more deceptive. Scalable oversight needed. 👇

"Existing benchmarks are either ad hoc or focus on these very specific and unrealistic capture-the-flag exercises" Daniel Kang introduces CVE-Bench, 1st benchmark for testing AI agents against real-world cybersecurity vulnerabilities. Success drops from 60% on artificial test to 10% on real ones.