Very honored to be one out of seven outstanding papers at this years' EMNLP :) Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them!

Presenting today our work "Unsupervised Word-level Quality Estimation Through the Lens of Annotator (Dis)agreement" at the Machine Translation morning session (Room A301, 11:45 China time). See you there! 🤗 Paper: aclanthology.org/2025.emnlp-m... Slides/video/poster: underline.io/events/502/s...

Here’s a custom feed for #EMNLP2025. Click the pin to save it to your home screen!

Flying out to @emnlpmeeting soon🇨🇳 I'll present our parametric CoT faithfulness work (arxiv.org/abs/2502.14829) on Wednesday at the second Interpretability session, 16:30-18:00 local time A104-105 If you're in Suzhou, reach out to talk all things reasoning :)

⏰ One week left to apply for the two PhD Fellowships in Trustworthy NLP and Explainable NLU! The two positions have a starting date in spring 2026. Check the original post for more details👇

📣Tomorrow at #COLM2025: 1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications... 2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally 1/3

If you're at COLM, check out various works by Ana and her group!

🤔What happens when LLM agents choose between achieving their goals and avoiding harm to humans in realistic management scenarios? Are LLMs pragmatic or prefer to avoid human harm? 🚀 New paper out: ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs🚀🧵

I won't be at COLM, so come see Yonatan talk about our work on estimating CoT faithfulness using machine unlearning! Check out the thread for the (many) other interesting works from his group 🎉

Here’s a #COLM2025 feed! Pin it 📌 to follow along with the conference this week!

Josip Juki\'c, Martin Tutek, Jan \v{S}najder Context Parametrization with Compositional Adapters https://arxiv.org/abs/2509.22158

Adi Simhi, Jonathan Herzig, Martin Tutek, Itay Itzhak, Idan Szpektor, Yonatan Belinkov ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs https://arxiv.org/abs/2510.00857

Opportunities to join my group in fall 2026: * PhD applications direct or via ELLIS @ellis.eu (ellis.eu/news/ellis-p...) * Post-doc applications direct or via Azrieli (azrielifoundation.org/fellows/inte...) or Zuckerman (zuckermanstem.org/ourprograms/...)

What's the right unit of analysis for understanding LLM internals? We explore in our mech interp survey (a major update from our 2024 ms). We’ve added more recent work and more immediately actionable directions for future work. Now published in Computational Linguistics!

🎓 Fully funded PhD in Trustworthy NLP at the UCPH & @aicentre.dk with @iaugenstein.bsky.social and me, @copenlu.bsky.social 📆 Application deadline: 30 October 2025 👀 Reasons to apply: www.copenlu.com/post/why-ucph/ 🔗 Apply here: candidate.hr-manager.net/ApplicationI... #NLProc #XAI #TrustworhyAI

🚨 Are you looking for a PhD in #NLProc dealing with #LLMs? 🎉 Good news: I am hiring! 🎉 The position is part of the “Contested Climate Futures" project. 🌱🌍 You will focus on developing next-generation AI methods🤖 to analyze climate-related concepts in content—including texts, images, and videos.

The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we @icepfl.bsky.social @ethz.ch @cscsch.bsky.social ) built Apertus.

🚨 EACL 2026 website is live and Call for Papers is out! 🚨 Join us at #EACL2026 (Rabat, Morocco 🇲🇦, Mar 24-29 2026) 👉 Open to all areas of CL/NLP + related fields. Details: 2026.eacl.org/calls/papers/ • ARR submission deadline: Oct 6, 2025 • EACL commitment deadline: Dec 14, 2025

- Fully funded PhD fellowship on Explainable NLU: apply by 31 October 2025, start in Spring 2026: candidate.hr-manager.net/ApplicationI... - Open-topic PhD positions: express your interest through ELLIS by 31 October 2025, start in Autumn 2026: ellis.eu/news/ellis-p... #NLProc #XAI

All your embarrassing secrets are training data (unless you are paying attention)

How many people would you estimate are currently actively publishing in ML research? From AAAI, which has ~29000 submissions: "There are 75,000+ unique submitting authors." NeurIPS had 25000 submissions. Is the number close to 300k? 500k?

Is there any information wrt. EMNLP limited registraton policy? I'm assuming registering authors should be safe, but idk about rest. Appreciate any information. Talking about this: "Given the expected popularity of EMNLP 2025, we may need to limit registration."

Thrilled that FUR was accepted to @emnlpmeeting.bsky.social Main🎉 In case you can’t wait so long to hear about it in person, it will also be presented as an oral at @interplay-workshop.bsky.social @colmweb.org 🥳 FUR is a parametric test assessing whether CoTs faithfully verbalize latent reasoning.

"Out of 13,048 reviewers..only 69 were deemed highly irresponsible..and enforcement was applied solely in those cases...These reviewers were contacted multiple times...as well as being personally contacted by the area chairs and senior area chairs, but still failed to fulfill them." 🫡🫡🫡

This year, EMNLP ended up desk rejecting ~100 papers. For more insight into the process, and potential future changes, please see this blog post from the PCs: 2025.emnlp.org/desk-rejecti... @christos-c.bsky.social @carolynrose.bsky.social @tanmoy-chak.bsky.social @violetpeng.bsky.social

Tomer Ashuach, Dana Arad, Aaron Mueller, Martin Tutek, Yonatan Belinkov CRISP: Persistent Concept Unlearning via Sparse Autoencoders https://arxiv.org/abs/2508.13650

Announcing a deep net interpretability talk series! Every week you will find new talks on recent research in the science of neural networks. The first few are posted: jackmerullo.bsky.social, Roy Rinberg, and me. At the @ndif-team.bsky.social Youtube Channel: www.youtube.com/@NDIFTeam

📢 Call for Papers! 📢 #BlackboxNLP 2025 invites the submission of archival and non-archival papers on interpreting and explaining NLP models. 📅 Deadlines: Aug 15 (direct submissions), Sept 5 (ARR commitment) 🔗 More details: blackboxnlp.github.io/2025/call/

Excited to spend the rest of the summer visiting @davidbau.bsky.social's lab at Northeastern! If you’re in the area and want to chat about interpretability, let me know ☕️

🎲 The 2nd edition of UncertaiNLP is coming to EMNLP 2025 in Suzhou! A venue for work on uncertainty-aware NLP, from Bayesian inference to decision-making under uncertainty. 🗓 Direct submissions due: Aug 15 🗓 ARR commitments due: Aug 29 Details: uncertainlp.github.io

🔈For the SoLaR workshop @COLM_conf we are soliciting opinion abstracts to encourage new perspectives and opinions on responsible language modeling, 1-2 of which will be selected to be presented at the workshop. Please use the google form below to submit your opinion abstract ⬇️

The report deadline was also extended to August 10th! Note that this is a final extension. We look forward to reading your reports! ✍️

The EPFL NLP lab is looking to hire a postdoctoral researcher on the topic of designing, training, and evaluating multilingual LLMs: docs.google.com/document/d/1... Come join our dynamic group in beautiful Lausanne!

Just 5 days left to submit your method to the MIB Shared Task at #BlackboxNLP! Have last-minute questions or need help finalizing your submission? Join the Discord server: discord.gg/n5uwjQcxPR

olmoTrace for connecting model generations to training data won Best Paper for System Demonstrations at #ACL2025!

Results deadline extended by one week! Following requests from participants, we’re extending the MIB Shared Task submission deadline by one week. 🗓️ New deadline: August 8, 2025 Submit your method via the MIB leaderboard!

📝 Technical report guidelines are out! If you're submitting to the MIB Shared Task at #BlackboxNLP, feel free to take a look to help you prepare your report: blackboxnlp.github.io/2025/task/

Just arrived in Vienna for @aclmeeting.bsky.social, reach out if you want to talk explainability or just grab a beer and relax from the busy schedule 🍻

The #ACL2025 #ACL2025NLP feed is up and running! It matches both hashtags and any posts from or mentions of @aclmeeting.bsky.social Pin it to your home 📌 and enjoy! bsky.app/profile/did:...

1️⃣ Spotlight: Gender Bias in How Public Figures are Referenced in News Quotes at Scale Wed 23 Jul CET - Talk: 09:45 - 10:30 - De Geerhallen - Poster: 13:30 - 14:30 - Atrium Contrary to earlier findings, we show that left-wing politicians use first names more often when referring to women!

Are you attending ICML? 👀 I'm sadly not, but if you are, you should check out the MIB 🕶️poster at 11AM: icml.cc/virtual/2025... The benchmark is used as the shared task at this year's @blackboxnlp.bsky.social (blackboxnlp.github.io/2025/task/) - there's still time to participate 🏆

We did a cool group project exploring diachronic embeddings for Croatian and found that (among other things) embeddings trained on later periods are more positive when plugged into models trained on earlier time periods. Check out the thread 🧵 & come talk to us in Vienna about this & other works 🍻

📣📣 New preprint alert!! Despite events in the world becoming bleaker, the news is… more positive? We conduct a diachronic study of word embeddings trained on 10M Croatian news articles spanning 25 years and find some surprising results! arxiv.org/abs/2506.13569

Missed a spot? If you have a pre-reviewed paper from ARR or COLM that focuses on the INTERPLAY between LM internals and behavior, there is a shortcut to presenting at our @colmweb.org workshop! ✨ Join us in Montréal! 🇨🇦 CfP: shorturl.at/sBomu OpenReview: shorturl.at/WwWhg #nlproc #interpretability

Working on feature attribution, circuit discovery, feature alignment, or sparse coding? Consider submitting your work to the MIB Shared Task, part of this year’s #BlackboxNLP We welcome submissions of both existing methods and new or experimental POCs!

New to mechanistic interpretability? The MIB shared task is a great opportunity to experiment: ✅ Clean setup ✅ Open baseline code ✅ Standard evaluation Join the discord server for ideas and discussions: discord.gg/n5uwjQcxPR

First time a judge has decided a case based on hallucinated case law in the US that I've encountered. caselaw.findlaw.com/court/ga-cou...

1/ 🚨NEW PAPER: "BriefMe: A Legal NLP Benchmark for Assisting with Legal Briefs", accepted to ACL Findings 2025! We introduce the first benchmark specifically designed to help LLMs assist lawyers in writing legal briefs 🧑‍⚖️ 📄 arxiv.org/abs/2506.06619 🗂️ huggingface.co/datasets/jw4...

One month to go! ⏰ Working on featurization methods - ways to transform LM activations to better isolate causal variables? Submit your work to the Causal Variable Localization Track of the MIB Shared Task!

Working on circuit discovery in LMs? Consider submitting your work to the MIB Shared Task, part of #BlackboxNLP at @emnlpmeeting.bsky.social 2025! The goal: benchmark existing MI methods and identify promising directions to precisely and concisely recover causal pathways in LMs >>