We are hiring in AI and NLP at Utah! Please apply if interested and share with folks you know are looking for faculty positions, especially if they like to ski, hike, climb, or bike! utah.peopleadmin.com/postings/190...

💫 On the heels of announcing 12 new faculty fellows last week, SCI's One-U Responsible AI Initiative is excited to add three new postdoctoral fellows to its team next year: rai.utah.edu/postdocs-dec...

KSL TV @ksl.com interviewed the members of my research group who recently discovered a rare piece of computing history: an old tape that might contain UNIX V4. ksltv.com/science-tech...

Make it make sense 😂

📣 Meet the 12 new faculty fellows joining SCI’s One-U Responsible AI Initiative. These professors are advancing AI research to solve real-world challenges, from protecting the West’s water supply to improving medical care to embedding ethics in AI education. bit.ly/rai-fac-26

Is it me or more reviewers are -strongly- opinionated about how you paper should be written? I feel like my responses in the past year or two are turning into arguments about why my writing is just fine

Current ChatGPT response format is such an eyesore, what’s up with all the lists and emojis

Luca Guadagnino making an OpenAI movie, what's happening 👀👀

I didn't submit to ICLR, but I'm pretty sure next week I'll see similar quality issues in ARR I feel like execs of major ML/AI conferences from ICLR/NeurIPS/ICML/AAAI, ACL/EMNLP, to CVPR should sit together and figure out a whole new strategy moving forward like 👇

When your husband is also an academic, so when you can't get him on a phone you shoot an email and it works every time 😂💀

We're surveying researchers about name changes in academic publishing. If you've changed your name and dealt with updating publications, we want to hear your experience. Any reason counts: transition, marriage, cultural reasons, etc. forms.cloud.microsoft/e/E0XXBmZdEP

@mclemcrew.bsky.social's CoLM spotlight is now available on YT! 🎵 youtu.be/w6LNmADnlNw?...

*Urgently* looking for emergency reviewers for the ARR October Interpretability track 🙏🙏 ReSkies much appreciated

Outstanding paper (5/7): "Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps" by Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasovic, and Yonatan Belinkov aclanthology.org/2025.emnlp-m... 6/n

𝙒𝙚'𝙧𝙚 𝙝𝙞𝙧𝙞𝙣𝙜 𝙣𝙚𝙬 𝙛𝙖𝙘𝙪𝙡𝙩𝙮 𝙢𝙚𝙢𝙗𝙚𝙧𝙨! KSoC: utah.peopleadmin.com/postings/190... (AI broadly) Education + AI: - utah.peopleadmin.com/postings/189... - utah.peopleadmin.com/postings/190... Computer Vision: - utah.peopleadmin.com/postings/183...

🎉 Congratulations to all #EMNLP2025 award winners 🎉 Starting with the ✨Best Paper award ✨: "Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index" by Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi aclanthology.org/2025.emnlp-m... 1/n

Thrilled to see this work recognized at #EMNLP2025! This framework and approach to measuring CoT faithfulness have been hugely influential for how I think about reasoning evaluation, and I'm so lucky to have worked with such brilliant collaborators. Huge credit to @mtutek.bsky.social

Very honored to be one out of seven outstanding papers at this years' EMNLP :) Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them!

Check out Martin's talk at #EMNLP2025 today (Wed)! If you care about CoT faithfulness, you 𝘮𝘶𝘴𝘵 read this paper. It introduces the first method for measuring CoT faithfulness that is not purely behavioral, but operates with the internals!

Go check Alex's poster today (Wed) in Suzhou! #EMNLP2025 I'm still so proud of our work (led by @lasha.bsky.social) on CondaQA, so we had to ask what would happen if we tried to create high-quality reasoning-over-text benchmarks now that LLMs are available. Turns out, we'd make an easier benchmark!

Flying out to @emnlpmeeting soon🇨🇳 I'll present our parametric CoT faithfulness work (arxiv.org/abs/2502.14829) on Wednesday at the second Interpretability session, 16:30-18:00 local time A104-105 If you're in Suzhou, reach out to talk all things reasoning :)

Sort of starting to believe that we really do need academic metrics that punish publishing too much

I'll be in Suzhou 🇨🇳 at #EMNLP this week presenting "What has been Lost with Synthetic Evaluation?" done with @anamarasovic.bsky.social & @lasha.bsky.social! 🎉 📍Findings Session 1 - Hall C 📅 Wed, November 5, 13:00 - 14:00 arxiv.org/abs/2505.22830

🧠 Can large language models build the very benchmarks used to evaluate them? In “What Has Been Lost with Synthetic Evaluation”, Ana Marasović (@anamarasovic.bsky.social) and collaborators ask what happens when LLMs start generating the datasets used to test their reasoning. (1/6🧵)

👉 Do large language models really reason the way their chain-of-thoughts suggest? This week on #WiAIRpodcast, we talk with Ana Marasović (@anamarasovic.bsky.social) about her paper “Chain-of-Thought Unfaithfulness as Disguised Accuracy.” (1/6🧵) 📄 Paper: arxiv.org/pdf/2402.14897

Can you trust your reward model alignment scores? New work presented today at the COLM Workshop on Socially Responsible Language Modelling Research led by Purbid Bambroo and in collaboration with @anamarasovic.bsky.social that probes LLM preference test sets for redundancy and inflated scores. 1/8

📣Tomorrow at #COLM2025: 1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications... 2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally 1/3

📣Tomorrow at #COLM2025: 1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications... 2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally 1/3

Sad: Can't go to CoLM because of immigration. Happy: Well, at least I can mountain bike during the fall break in prime SLC MTB weather. Sad: Comes down with a cold. ☹️☹️☹️☹️☹️☹️

I had a great time chatting with Jekaterina and Malikeh. This episode is like a tour of all the things I've been studying lately!

🎙️ New Women in AI Research episode out now! This time, we sit down with @anamarasovic.bsky.social to unpack some of the toughest questions in AI explainability and trust. 🔗 Watch here → youtu.be/xYb6uokKKOo

Happening today! #COLM2025

In #Interplay25 workshop, Friday ~11:30, I'll present on measuring *parametric* CoT faithfulness on behalf of @mtutek.bsky.social , who couldn't travel: bsky.app/profile/mtut... Later that day we'll have a poster on predicting success of model editing by Yanay Soker, who also couldn't travel

I won't be at COLM, so come see Yonatan talk about our work on estimating CoT faithfulness using machine unlearning! Check out the thread for the (many) other interesting works from his group 🎉

Honored 🎷🎸🥁 𝗠𝗶𝘅𝗔𝘀𝘀𝗶𝘀𝘁 🎷🎸🥁 was selected as the #COLM2025 oral spotlight. Go check out @mclemcrew.bsky.social's 𝐭𝐚𝐥𝐤 on 𝐖𝐞𝐝 (𝐎𝐜𝐭 𝟖) at 𝟑:𝟑𝟎𝐩𝐦 in 517BC and 𝐩𝐨𝐬𝐭𝐞𝐫 from 𝟒:𝟑𝟎-𝟓:𝟑𝟎 in 710!

Mega stoked to attend #COLM25 this week and present our work, MixAssist, on Wednesday! @anamarasovic.bsky.social sadly can't make it 😭, but hit me up if you'd like to chat about audio language models, music mixing, or anything else regarding music and audio!

Our new guest at #WiAIRpodcast is @anamarasovic.bsky.social (Asst prof @ University of Utah , Ex @ Allen AI). We'll talk with her about faithfulness, trust and robustness in AI. The episode is coming soon, don't miss: www.youtube.com/@WomeninAIRe... #WiAIR #NLProc

Pls help spread the word: We are hiring! 🌲☕✨ @ischool.uw.edu is recruiting two tenure-track assistant profs in AI. Areas include human-AI collaboration, AI & society, comp social science, NLP, and AI applications. Apply by Nov 15, 2025: apply.interfolio.com/171020 Happy to answer questions!

Beyond excited to share that I’ll be joining Universal Audio as an LLM Applications Engineer! So stoked to continue working on co-creative AI elements of music production in this role 🎹 S/O to @anamarasovic.bsky.social for teaching me all things NLP these past few years!

Learned a ton working on this survey. Hope it's helpful for you too, check it out! Feedback welcome 🙏 kennethmarino.com/computeruse/...

Super excited that the Computer Use survey I've been working on w/ @anamarasovic.bsky.social for a while now is ready! Originally we were planning on a more traditional survey paper but as more surveys came out we decided on an interactive website survey.

Check @mtutek.bsky.social's thread!

Thrilled that FUR was accepted to @emnlpmeeting.bsky.social Main🎉 In case you can’t wait so long to hear about it in person, it will also be presented as an oral at @interplay-workshop.bsky.social @colmweb.org 🥳 FUR is a parametric test assessing whether CoTs faithfully verbalize latent reasoning.

Welp #acl2025 #acl2025nlp

Now in hall X4, board 407! #ACL2025 #ACL2025NLP

A delayed flight had another plan for my evening 😩

Arriving to #ACL2025 #ACL2025NLP in a few hours! See you at the welcome reception & catch me at the poster session on 𝐓𝐮𝐞𝐬𝐝𝐚𝐲 (𝐉𝐮𝐥𝐲 𝟐𝟗) 𝐚𝐭 𝟏𝟎:𝟑𝟎𝐚𝐦, where Jesse will present our work introducing new tasks for supporting legal brief writing: arxiv.org/abs/2506.06619

The box and I are in similar states

My first audio AI paper, thanks to @mclemcrew.bsky.social who introduced me to a whole new world of co-creative music producing! Getting to work with students who bring their unique passion into PhD work is one of the best perks of a professor's job. Check more in the thread👇🏻 Soon at #colm2025

Reunited with bikes in Bishkek 😮‍💨 Off to 10 days of biking in Kyrgyzstan > Croatia > Vienna (ACL) > more Croatia