🤔 How do we know if a reward model is truly good? In our last #WiAIR episode, Valentina Pyatkin (AI2 & University of Washington) introduced RewardBench 2—a harder, cleaner benchmark for reward models in post-training. (1/8🧵)

💥 Behind every success is a story of rejection. Persistence, curiosity, and resilience are what truly drive AI careers. 🚀 Don't miss the full episode: 🎬 YouTube: youtube.com/watch?v=DPhq... 🎙 Spotify: open.spotify.com/episode/7aHP...

💡 Are LLMs truly good at precise instruction following, or just overfitting to benchmarks? In our latest WiAIR episode, we sit down with Valentina Pyatkin (@valentinapy.bsky.social) from @ai2.bsky.social and UW to discuss her paper: “Generalizing Verifiable Instruction Following”. (1/7🧵)

Tulu 3 isn’t just a model - it’s the ecosystem: data, recipes, benchmarks, and RLVR. Valentina Pyatkin breaks down how smart data mixing & filtering shaped its performance. Don't miss the full episode: 🎬 YouTube: youtube.com/watch?v=DPhq... 🎙 Spotify: open.spotify.com/episode/7aHP...

🚀 Can open science beat closed AI? Tülu 3 makes a powerful case. In our new #WiAIRpodcast, we speak with Valentina Pyatkin (@valentinapy.bsky.social) of @ai2.bsky.social and the University of Washington about a fully open post-training recipe—models, data, code, evals, and infra. #WomenInAI 1/8🧵

🚀 New WiAIR Podcast Episode! Can open-source LLMs really outperform closed ones like Claude 3.5? 🤔 We asked Valentina Pyatkin (AI2, UW) — and you'll be interested to hear her answers #NLProc #WiAIR #WiAIRpodcast

🔥 Ready for the new episode? Here’s a sneak peek of what’s coming next on the #WiAIR Podcast ✨ Full episode drops soon - don't miss it! 🔔 Subscribe here: 🎬 YouTube: www.youtube.com/@WomeninAIRe... 🎙️ Spotify: open.spotify.com/show/51RJNlZ... 🎧 Apple Podcasts: podcasts.apple.com/ca/podcast/w...

"𝐋𝐋𝐌 𝐏𝐨𝐬𝐭-𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠: 𝐎𝐩𝐞𝐧 𝐒𝐜𝐢𝐞𝐧𝐜𝐞 𝐓𝐡𝐚𝐭 𝐏𝐨𝐰𝐞𝐫𝐬 𝐏𝐫𝐨𝐠𝐫𝐞𝐬𝐬 " 🎙️ On Sept 17, the #WiAIRpodcast speaks with @valentinapy.bsky.social (@ai2.bsky.social & University of Washington) about open science, post-training, mentorship, and visibility #WiAIR #NLProc

💡 Should LLMs reason in natural language or symbolic language? Why not both? In our latest WiAIR episode, we sit down with Simeng Han to discuss HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning. (1/8 🧵) #WiAIR

LLMs are still black boxes — but two research directions stand out: ✨ Natural language explanations 🔎 Mechanistic interpretability Both reshape how we think about trust in AI. 💬 Which approach do you believe has more impact — and why? #AI #LLM #Interpretability #Explainability #WiAIR #WiAIRpodcast

🚨 Don’t just grade answers—grade reasoning. New WiAIR episode with Simeng Han (Yale) on P-FOLIO—human-written proofs that evaluate how LLMs reason, not only what they answer. #WiAIR #AIResearch (1/7)

Should AI mimic human reasoning—or take inspiration from it? Simeng Sophia Han (Yale PhD & Meta FAIR) explains why the most powerful AI models focus on efficiency, creativity, and generalizability, rather than copying humans. Full episode👇 youtube.com/watch?v=IalF...

💡 Do LLMs solve with creative insight, or just brute force? BRAINTEASER, a benchmark of expert-authored math and logic puzzles that test not just accuracy but also the reasoning process. #LLMs #AIReasoning (1/8)

AI research thrives on diverse perspectives. Simeng reminds us that mentorship from psychology, social sciences, & other fields can shape our growth in unexpected ways. 🔍 Has your most impactful mentor been within AI, or beyond it? Full episode👇 www.youtube.com/watch?v=IalF...

🚀 New WiAIR podcast episode is out! Guest: Sophia Simeng Han (Yale PhD, Meta FAIR, ex-DeepMind/AWS). We dive into: • Evaluating LLM reasoning beyond accuracy • Lessons from human reasoning chains • CogSci insights for AI #WiAIRpodcast #WiAIR

✨ Accuracy isn’t enough. To advance LLMs, we need to look beyond outputs & examine reasoning traces. Full convo with Simeng Han dropping Aug 27 on #WiAIRPodcast: 🎬 YouTube: www.youtube.com/@WomeninAIRe... 🎙️ Spotify: open.spotify.com/show/51RJNlZ...

What does it mean for an AI to reason? In our upcoming #WiAIR podcast (Aug 27), we chat with Simeng Sophia Han (PhD student at @yale.edu, intern at Meta FAIR) about LLM reasoning, human reasoning chains, cognitive science insights, and what makes AI outputs trustworthy. 🎧 Episode drops next week!

We rarely know what data trains today’s most powerful AI models. 📦 Abhilasha Ravichander breaks down why training data transparency matters—and how tools for indexing & surfacing evidence can reshape trust in AI.

💡 What do we really know about the training data behind today’s proprietary LLMs like GPT-4? Almost nothing. For legal, proprietary & reputational reasons, companies release few (if any) details about this “secret ingredient.” (1/7)

Abhilasha Ravichander shares how she learned to be assertive while staying true to her kindness, and why letting go of constant self-monitoring gave her more energy for what matters most 💪✨ Stay tuned: www.youtube.com/@WomeninAIRe... #ResearchLife #Inspiration

🌟 In our last episode, we sat down with Dr. Abhilasha Ravichander (@lasha.bsky.social), an inspiring AI researcher who will be joining the Max Planck Institute for Software Systems as faculty in Fall 2025. #WiAIR #WomenInAI #TrustworthyAI #NLP #MaxPlanckInstitute (1/3)

Early struggles and rejection are normal in academia. Your value as a scientist is not about how quickly things happen, it is about your persistence and passion. Keep going 💪✨ Stay tuned: www.youtube.com/@WomeninAIRe... #Science #Research #AcademicLife #WiAIRpodcast

We’re thrilled to congratulate Dr. Abhilasha Ravichander (@lasha.bsky.social) and her team for receiving the Outstanding Paper Award at #acl2025 for their work titled "HALoGEN: Fantastic LLM Hallucinations and Where to Find Them"! 🏆✨ #ACL #LLMs #Hallucination #WiAIR #WomenInAI

🎙️ New Women in AI Research episode out now! We speak with Dr. Abhilasha Ravichander about: – LLM hallucination types – Benchmarks WildHallucinations & HALoGEN – Machine unlearning + memorization probes – Responsible AI & transparent systems 🎧 Listen here: youtu.be/QPp0cJNBbL8 #LLMHallucinations

🤖 Can LLMs know when to be factual vs. creative? 🎙️ In this short clip, Abhilasha Ravichander explores one of the hardest challenges in LLM behavior: adaptability. 📅 Full #WiAIRpodcast episode drops in 2 days!

🎙️ New Women in AI Research #WiAIR episode coming Aug 6! We talk to @lasha.bsky.social about LLM Hallucination, her award-winning HALoGEN benchmark, and how we can better evaluate hallucinations in language models. 👇 What’s inside: 1/

Our guest Dieuwke Hupkes questions the lack of accountability in academic peer review. Why do bad reviewers get off the hook? Could consequences - like limiting their ability to submit - actually improve the system? What's your take on this?

🧠 Do scaling laws in LLMs apply equally across skills, or do knowledge and reasoning scale differently? A new study with Dieuwke Hupkes (Meta AI) shows that scaling laws are skill‑dependent. 👇 (1/8) #WiAIR #AIResearch

Trust in AI starts with generalization. Dieuwke Hupkes (#MetaAI) explains why it's critical to know when you can count on your model, and when to be cautious. Full episode: 🎥 Youtube: youtu.be/CuTWIW1JcsA?... 🎙️ Spotify: open.spotify.com/episode/0KSR... #LLMs #TrustworthyAI

Can models truly understand meaning—or are they just good at form? In this WiAIR episode, we speak with Dieuwke Hupkes about her new paper: From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency 🧵 (1/8)

Why is compositional generalization still such a hard problem for LLMs? Compositionality requires mapping between two distinct representational spaces — often syntax and semantics. But… 1/ #MetaAI #AIresearch #Compositionality

🧠 What does it really mean for an LLM to generalize? And are we even measuring it right? In the latest #WiAIR episode, we talk with Dr. Dieuwke Hupkes, a Research Scientist at Meta AI, about one of the most comprehensive reviews of generalization in NLP. 👇🧵 (1/8)

🎙️ In our latest #WiAIR episode, we chat with Dieuwke Hupkes from Meta AI on: • Generalization in language models • Consistency & alignment in #LLM • Research struggles, peer reviews, & balance Plus: a surprising productivity hack involving pole dancing. 👀 Tune in: youtu.be/CuTWIW1JcsA

LLM Consistency: The Hidden Clue to True Generalization? When a model gets Paris right in one language but not another, that inconsistency isn’t random—it’s revealing. Here’s a preview from our upcoming episode (dropping in 2 days). #AIResearch #LLMs #MultilingualAI #WiAIRpodcast #WiAIR

How do we know if a language model really generalizes—or is just repeating patterns it’s memorized? Let’s talk about consistency, contamination, and the future of trustworthy AI. 👇 🎧 New #WiAIR Podcast w/ Dieuwke Hupkes drops July 16! 1/ #WiAIRpodcast

Are we running out of training data for large language models? 🤖📉 AI researcher Wanru Zhao thinks so—and the implications are huge. /1 #LLM #TrainingData #FutureOfAI

🚀 The newest WiAIR episode spotlights Photon—Wanru Zhao’s federated system that lets GPUs on 0.8-8 Gbps links train decoder-only LLMs up to 7 B parameters faster than 400 Gbps clusters. 🌍⚡️ (1/7)

🇬🇧 vs 🇨🇦 AI research cultures — what’s the difference? Wanru Zhao breaks it down from her experiences in the UK and Canada. /1 #AIResearch #WomenInAI #AcademicCulture #VectorInstitute #PhDLife

🎙️ I don’t need to be someone that’s not me anymore. In AI research, finding your people can be life-changing. Wanru Zhao shares how community helped her embrace authenticity. 💜 Watch this clip👇 #WiAIR #WomenInAI #AuthenticityMatters #TechCommunity #WiAIRpodcast

🔐 Can multilingual LLMs be fine-tuned collaboratively across regions without sharing private data or full models? Wanru Zhao explores this in their ICLR 2024 paper: “Breaking Physical and Linguistic Borders” 🧵(1/8)

What if we stopped burying failed experiments in science? And started learning from them? 📽️ Watch this short clip 👇 #PhDLife #OpenScience #AcademicCulture

🚀 Can Large Language Models Learn Collaboratively without Ever Sharing Private Data? Wanru Zhao’s NeurIPS 2024 paper introduces CLUES — a privacy-first approach to collaborative data selection. 🧵 Here’s how it works: 👇 🔗 proceedings.neurips.cc/paper_files/...

🎙️ New WiAIR Podcast Ep is LIVE! We dive deep into the future of decentralized AI, scaling beyond scaling laws, and building inclusive, multilingual LLMs with Wanru Zhao. #WiAIR #DecentralizedAI #FederatedLearning #WomenInAI

🎙️ A new episode of the Women in AI Research podcast lands in 2 days! We’re joined by Wanru Zhao - PhD at Cambridge + visiting researcher at Vector - who unpacks why scaling LLMs isn’t as simple as it used to be. Sneak peek below 👇 1/

🎧 New @wiair.bsky.social podcast episode drops Wednesday, June 25th! We speak with ‪@wanru.bsky.social‬, a PhD student at @cam.ac.uk‬, visiting researcher at ‪@vectorinstitute.ai‬, and former intern at ‪@awscloud.bsky.social‬ & ‪@msftresearch.bsky.social‬. /1 #WiAIR #WomenInAI #AIResearch

AI in healthcare isn’t just exciting—it’s risky. 🎥 In this clip from our last #WiAIR podcast episode, we break down 3 major risks: ⚠️ Overreliance 🧠 Lack of interpretability 📉 Hidden bias 1/ #AIethics #HealthTech #WomenInAI

In our latest #WiAIR episode, we explore a fascinating paper, "Using Chain-of-Thought Prompting for Interpretable Recognition of Social Bias," co-authored by our guest Dr. Faiza Khan Khattak /1

Does AI have ego issues? 😅 When it answers first, it clings to wrong answers—then invents reasoning to defend itself. But if you ask it to reason first, it gets it right. Human-like? You decide. #AI #LLM #Reasoning #ArtificialIntelligence #TechShorts #AIEthics #WiAIR

🤖 Why do many high-performing clinical ML models never make it into real hospitals? 🎙️ In this WiAIR episode, we talk with Dr. Faiza Khan Khattak about MLHOps, a framework for integrating ML into healthcare systems operationally, ethically, and within clinical workflows. #WiAIR #MLHOps #ClinicalAI

“They trusted me—but I still saw bias.” A powerful story on how *unconscious bias* affects even trusted women leaders. It’s subtle. It’s real. It’s time we talk about it. #WomenInTech #BiasInAI #Leadership #Inclusion #WiAIR