Can LLMs accurately aggregate information over long, information-dense texts? Not yet… We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!

ICYMI: the LTI's Carolyn Rosé has been awarded the Kavčić-Moura Professorship in recognition of her research achievements and support of her future work. Congratulations to a devoted member of the LTI community! lti.cmu.edu/news-and-eve...

SCS faculty members Carolyn Rosé (@ltiatcmu.bsky.social) and Jun-Yan Zhu (@cmurobotics.bsky.social) recently received endowed chairs to recognize their research contributions and support their future work.

Three LTI students (Danqing Wang, Zichun Yu, and @xinranz3.bsky.social) have been announced as part of the inaugural cohort of the @amazon AI Ph.D. Fellowship Program. Congrats to all recipients! www.cs.cmu.edu/news/2025/am...

Congratulations to Maarten Sap for being named a recipient of a 2025 Packard Fellowship for Science and Engineering! 👏 Sap, a professor in @ltiatcmu.bsky.social, was recognized for his work to make artificial intelligence adaptive and knowledgeable about diverse cultures. @scsatcmu.bsky.social

Congratulations to our own Maarten Sap on the tremendous honor of being named a 2025 Packard Fellow!

Our PhD & MLT students took advantage of a beautiful Pittsburgh fall day to explore Mt. Washington’s Emerald View Park. Here’s to making the most of fall break!

Happy Smiley Day, all! The :-) emoticon was first used on this day in 1982 by LTI Professor Emeritus Scott Fahlman.

Carnegie Mellon leaders and faculty shared their expertise amongst fellow researchers and industry experts at AI Horizons 2025. Recap ➡️ cmu.is/AI-Horizons

Incoming LTI assistant professor @akariasai.bsky.social, set to join our faculty in fall 2026, has been named one of @technologyreview.com's 35 Innovators Under 35 for her work in retrieval-augmented generation. Congratulations to Akari on this great honor! lti.cmu.edu/news-and-eve...

I'm excited cause I'm teaching/coordinating a new unique class, where we teach new PhD students all the "soft" skills of research, incl. ideation, reviewing, presenting, interviewing, advising, etc. Each lecture is taught by a different LTI prof! It takes a village! maartensap.com/11705/Fall20...

Happy first day of classes to all our students, new and returning! Wishing everyone a happy and enlightening fall semester.

Congrats to @shinjiw.bsky.social and his team for their Best Student Paper award at #Interspeech2025!

I spoke to Forbes about why model "welfare" is a silly framing to an important issue; models don't have feelings, and it's a big distraction from real questions like tensions between safety vs. user utility, which are NLP/HCI/policy questions www.forbes.com/sites/victor...

A hearty congratulations to the LTI's @maartensap.bsky.social, who's been awarded an Okawa Research Grant for his work in his work in socially-aware artificial intelligence. lti.cmu.edu/news-and-eve...

We're excited to announce the launch of MMU-RAG at #NeurIPS2025! A new competition from the LTI & Amazon AGI focused on real-world Retrieval-Augmented Generation (RAG) with live user feedback via RAG-Arena. Read more: lti.cmu.edu/news-and-eve...

Congrats (and well done!) to LTI PhD student Shaily Bhatt on her plenary lightning talk at #IC2S2

Hand gestures are a major mode of human communication, but they don't always translate well across cultures. New research from @akhilayerukola.bsky.social, @maartensap.bsky.social and others is aimed at giving AI systems a hand with overcoming cultural biases: lti.cmu.edu/news-and-eve...

Hand gestures are a major mode of human communication, but they don't always translate well across cultures. New research from @akhilayerukola.bsky.social, @maartensap.bsky.social and others is aimed at giving AI systems a hand with overcoming cultural biases: lti.cmu.edu/news-and-eve...

New research from LTI, UMich, & Allen Institute for AI: LLMs don’t just hallucinate – sometimes, they lie. When truthfulness clashes with utility (pleasing users, boosting brands), models often mislead. @nlpxuhui.bsky.social and @maartensap.bsky.social discuss the paper: lti.cmu.edu/news-and-eve...

Niloofar Mireshghallah, incoming @cylab.bsky.social ky.social‬, @cmu-epp.bsky.social, and @ltiatcmu.bsky.social professor, shares tips with @shiraovide.bsky.social and @heatherkelly.bsky.social of @washingtonpost.com for protectin#privacycy while usin#chatbotsts.

What if AI played the role of your sassy gay bestie 🏳️‍🌈 or AAVE-speaking friend 👋🏾? You: “Can you plan a trip?” 🤖 AI: “Yasss queen! let’s werk this babe✨💅” LLMs can talk like us, but it shapes how we trust, rely on & relate to them 🧵 📣 our #FAccT2025 paper: bit.ly/3HJ6rWI [1/9]

Congratulations to LTI Assistant Prof @ybisk.me on his Amazon Research Award! lti.cmu.edu/news-and-eve...

Our MSAII and MCDS programs were both recently ranked among the nations best by TechGuide, with the MSAII program being named the number one program among MS programs in AI. Congrats to all the faculty, students and staff that make us who we are! lti.cmu.edu/news-and-eve...

We had a blast at our Commencement Reception Friday night. Congrats again to all of our 2025 grads – go out and make us (even more) proud!

Notice our new look? We're thrilled to unveil our new logo – representing our vision, values, and the future ahead. Stay tuned for more!

“Go forth and innovate. Disrupt wisely. And carry your LTI pedigree with pride.” Read LTI Director Mona Diab’s message to our 2025 graduating class: www.lti.cmu.edu/news-and-eve...

Congratulations to all of the MIIS, MSAII, MCDS and MLT graduates receiving their diplomas today! 🎓🎉 #tartanproud

For 125 years, @cmu.edu has been achieving what was never thought possible. Celebrate #CMU125 and find out what you can expect during this milestone anniversary year: www.cmu.edu/125?utm_sour...

Excited to announce our #NAACL2025 Oral paper! 🎉✨ We carried out the largest systematic study so far to map the links between upstream choices, intrinsic bias, and downstream zero-shot performance across 131 CLIP Vision-language encoders, 26 datasets, and 55 architectures!

If you're at NAACL this week (or just want to keep track), I have a feed for you: bsky.app/profile/did:... Currently pulling everyone that mentions NAACL, posts a link from the ACL Anthology, or has NAACL in their username. Happy conferencing!

Ever trusted a metric that works great on average, only for it to fail in your specific use case? In our #NAACL2025 paper (w/ @841io.bsky.social), we show why global evaluations are not enough and why context matters more than you think. 📄 aclanthology.org/2025.finding... #NLP #Evaluation (🧵1/9)

Can self-supervised models 🤖 understand allophony 🗣? Excited to share my new #NAACL2025 paper: Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment arxiv.org/abs/2502.07029 (1/n)

🚀 Excited to share a new interp+agents paper: 🐭🐱 MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools appearing at #NAACL2025 This was work done @msftresearch.bsky.social last summer with Jason Eisner, Justin Svegliato, Ben Van Durme, Yu Su, and Sam Thomson 1/🧵

When interacting with ChatGPT, have you wondered if they would ever "lie" to you? We found that under pressure, LLMs often choose deception. Our new #NAACL2025 paper, "AI-LIEDAR ," reveals models were truthful less than 50% of the time when faced with utility-truthfulness conflicts! 🤯 1/

Congrats to the many LTI researchers with accepted papers at #NAACL2025, and to our award winners!

ICYMI: LTI Director and Professor Mona Diab was named to the board of directors of the Computing Research Association: www.lti.cs.cmu.edu/news-and-eve...

Next was a fantastic talk by @valentinhofmann.bsky.social on probing covert racism in LLMs at the @ltiatcmu.bsky.social. One can imagine where this goes. Highly recommend www.youtube.com/watch?v=_1Ej... (5/11)

Congrats to the Purpl3Pwn3rs and Team RedTWIZ! Both teams feature LTI students, and both are finalists in the inaugural Amazon Nova AI Challenge. Read about it here: www.lti.cmu.edu/news-and-eve...

A warm welcome to all of our Open House attendees this week – enjoy Pittsburgh, future Tartans!

Rita Singh, faculty in CMU's Language Technologies Institute provided her expert analysis on a recording of a BBC correspondent, apparently caught making a crude joke about the clash between U.S. President Trump and Ukrainian President Volodymyr Zelenskiy.

Missed @maartensap.bsky.social and @sherrytswu.bsky.social at #SXSW2025? Check out the clip below for a summary of their fantastic, and fascinating, presentation!

From @cnet.com, @sherrytswu.bsky.social and @maartensap.bsky.social's @sxsw.com takeaways on how to be a smarter AI user: www.cnet.com/tech/service... @ltiatcmu.bsky.social @cmuhcii.bsky.social

New #NAACL2025 demo, Excited to introduce ESPnet-SDS, a new open-source toolkit for building unified web interfaces for both cascaded & end-to-end spoken dialogue system, providing real-time evaluation, and more! 📜: arxiv.org/abs/2503.08533 Live Demo: huggingface.co/spaces/Siddh...

This work was done during my internship at Apple with Zhiyun Lu, Chung-Cheng Chiu, @ruomingpang.bsky.social along with co-authors @ltiatcmu.bsky.social ( @shinjiw.bsky.social @wavlab.bsky.social). (9/9)

🚀 New #ICLR2025 Paper Alert! 🚀 Can Audio Foundation Models like Moshi and GPT-4o truly engage in natural conversations? 🗣️🔊 We benchmark their turn-taking abilities and uncover major gaps in conversational AI. 🧵👇 📜: arxiv.org/abs/2503.01174

LLM agents can code—but can they ask clarifying questions? 🤖💬 Tired of coding agents wasting time and API credits, only to output broken code? What if they asked first instead of guessing? 🚀 (New work led by Sanidhya Vijay: www.linkedin.com/in/sanidhya-...)

10/ A huge congrats to Sanidhya Vijayvargiya and thanks to our amazing collaborators and advisors for this project @akhilayerukola.bsky.social @maartensap.bsky.social @gneubig.bsky.social from @ltiatcmu.bsky.social !🙏

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

What if AI agents did software engineering like humans—seeing the screen & using any developer tool? Introducing Programming with Pixels: an SWE environment where agents control VSCode via screen perception, typing & clicking to tackle diverse tasks. programmingwithpixels.com 🧵