Incredibly honored to serve as #EMNLP 2026 Program Chair along with @sunipadev.bsky.social and Hung-yi Lee, and General Chair @andre-t-martins.bsky.social. Looking forward to Budapest!! (With thanks to Lisa Chuyuan Li who took this photo in Suzhou!)

Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP! Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!

Think your LLMs “understand” words like although/but/therefore? Think again! They perform at chance for making inferences from certain discourse connectives expressing concession

🚨 Does your LLM really understand code -- or is it just really good at remembering it? We built **PLSemanticsBench** to find out. The results: a wild mix. ✅The Brilliant: Top reasoning models can execute complex, fuzzer-generated programs -- even with 5+ levels of nested loops! 🤯 ❌The Brittle: 🧵

Find my students and collaborators at COLM this week! Tuesday morning: @juand-r.bsky.social and @ramyanamuduri.bsky.social 's papers (find them if you missed it!) Wednesday pm: @manyawadhwa.bsky.social 's EvalAgent Thursday am: @anirudhkhatry.bsky.social 's CRUST-Bench oral spotlight + poster

We’re hiring faculty as well! Happy to talk about it at COLM!

Can we quantify what makes some text read like AI "slop"? We tried 👇

I’m at #COLM2025 from Wed with: @siyuansong.bsky.social Tue am introspection arxiv.org/abs/2503.07513 @qyao.bsky.social Wed am controlled rearing: arxiv.org/abs/2503.20850 @sashaboguraev.bsky.social INTERPLAY ling interp: arxiv.org/abs/2505.16002 I’ll talk at INTERPLAY too. Come say hi!

On my way to #COLM2025 🍁 Check out jessyli.com/colm2025 QUDsim: Discourse templates in LLM stories arxiv.org/abs/2504.09373 EvalAgent: retrieval-based eval targeting implicit criteria arxiv.org/abs/2504.15219 RoboInstruct: code generation for robotics with simulators arxiv.org/abs/2405.20179

Traveling to my first @colmweb.org🍁 Not presenting anything but here are two posters you should visit: 1. @qyao.bsky.social on Controlled rearing for direct and indirect evidence for datives (w/ me, @weissweiler.bsky.social and @kmahowald.bsky.social), W morning Paper: arxiv.org/abs/2503.20850

All of us (@kanishka.bsky.social @kmahowald.bsky.social and me) are looking for PhD students this cycle! If computational linguistics/NLP is your passion, join us at UT Austin! For my areas see jessyli.com

Can AI aid scientists amidst their own workflows, when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring? Check out @sebajoe.bsky.social’s feature on ✨AstroVisBench:

📣 NEW HCTS course developed in collaboration with @tephi-tx.bsky.social: AI in Health Communication 📣 Explore responsible applications and best practices for maximizing impact and building trust with @utaustin.bsky.social experts @jessyjli.bsky.social & @mackert.bsky.social. 💻: rebrand.ly/HCTS_AI

long range narrative understanding, even basic fact checking that humans easily get near perfect on, has barely improved in LMs over years novelchallenge.github.io

🤖 🧠 NEW PAPER ON COGSCI & AI 🧠 🤖 Recent neural networks capture properties long thought to require symbols: compositionality, productivity, rapid learning So what role should symbols play in theories of the mind? For our answer...read on! Paper: arxiv.org/abs/2508.05776 1/n

I agree this thread's headline claim seems premature. Let me add our recent ACL Findings paper, with Dexter Ju and @hagenblix.bsky.social, which found syntactic simplification in at least some LMs, in a novel domain regeneration setting: aclanthology.org/2025.finding...

The Echoes in AI paper showed quite the opposite with also a story continuation setup. Additionally, we present evidence that both *syntactic* and *discourse* diversity measures show strong homogenization that lexical and cosine used in this paper do not capture.

Tuesday at #ACL2025: Jan will be presenting this from 4-5:30pm in x4/x5! Turns out content selection in LLMs are highly consistent with each other, but not so much with their own notion of importance or with human’s…

Looking forward to attending #cogsci2025 (Jul 29 - Aug 3)! I’m especially excited to meet students who will be applying to PhD programs in Computational Ling/CogSci in the coming cycle. Please reach out if you want to meet up and chat! Email is the best way, but DM also works if you must! quick🧵:

If you’re heading to ICML, check out Hongli’s work on context-specific alignment!

Check out this new opinion piece from Sebastian and Lily! We have really powerful AI systems now, so what’s the bottleneck preventing the wider adoption of fact checking systems, in high stakes scenarios like medicine? It’s how we define the tasks 👇

We have very good frameworks for cooperative dialog… but how about the opposite? @asher-zheng.bsky.social’s new paper takes a game-theoretic view and develops new metrics to quantify non-cooperative language ♟️ Turns out LLMs don’t have the pragmatic capabilities to perceive these…

Congrats to Sebastian et al. This is our first fully within-CosmicAI research project, setting the foundation for our future work.

Is AI ready to play a real role in science? This work with @nsfsimonscosmicai.bsky.social evaluates LLMs targeting the implementation of scientific workflows, and the scientific utility of visualizations from LLM-generated code -- and the answer is not yet, even with the best SOTA models 👇

Super thrilled that @kanishka.bsky.social is going to join UT Linguistics as our newest computational linguistics faculty member -- looking forward to doing great research together! 🧑‍🎓Students: Kanishka is a GREAT mentor -- apply to be his PhD student in the upcoming cycle!!

To appear #ICML2025!! 🎉

Where are those implicit, nuanced, and specific criteria in the land of LLM evaluation? Manya’s EvalAgent gets them!

We finally have a way to quantify that "LLM vibe" you feel about generated texts... There *are* tangible discourse similarities across models! Check out QUDsim👇

Have that eerie feeling of déjà vu when reading model-generated text 👀, but can’t pinpoint the specific words or phrases 👀? ✨We introduce QUDsim, to quantify discourse similarities beyond lexical, syntactic, and content overlap.

Team member highlight! Jessy Li @jessyjli.bsky.social associate professor of computational linguistics and NLP at @utaustin.bsky.social's Linguistics department, will be the AI lead of the Explorable Universe team. The team's goal is to develop generative AI models for astronomy research. #cosmicai

🌟Job ad🌟 We (@gregdnlp.bsky.social, @mattlease.bsky.social and I) are hiring a postdoc fellow within the CosmicAI Institute, to do galactic work with LLMs and generative AI! If you would like to push the frontiers of foundation models to help solve myths of the universe, please apply!

Do you want to know what information LLMs prioritize in text synthesis tasks? Here's a short 🧵 about our new paper, led by Jan Trienes: an interpretable framework for salience analysis in LLMs. First of all, information salience is a fuzzy concept. So how can we even measure it? (1/6)

🚨 Do LLMs fall for spin in medical literature? 🤔 In our new preprint, we find that LLMs are susceptible to biased reporting of clinical treatment benefits in abstracts—more so than human experts. 📄🔍 [1/7] Full Paper: arxiv.org/abs/2502.07963 🧵👇

The principles that LLMs align with should be specific to the task at hand! Check out @hongli-zhan.bsky.social’s latest work 👇

1. Hey NAACL #NLP #NLProc members please VOTE for the 25-26 exec! We have an amazing lineup of candidates, see statements here naacl.org/elections/20... How to vote: current members look for an email from Election Runner!

A starter pack for the NLP and Computational Linguistics researchers at UT Austin! go.bsky.app/75g9JLT

We at UT Linguistics are hiring for 🔥 2 faculty positions in Computational Linguistics! Assistant or Associate professors, deadline Dec 1. UT has a super vibrant comp ling & #nlp community!! Apply here 👉 apply.interfolio.com/158280

All the ACL chapters are here now: @aaclmeeting.bsky.social @emnlpmeeting.bsky.social @eaclmeeting.bsky.social @naaclmeeting.bsky.social #NLProc

Chuffed our paper @kanishka.bsky.social on using “controlled rearing” to learn the “a beautiful five days in Miami” arxiv.org/abs/2403.19827 construction won an EMNLP Outstanding Paper Award! And delighted that the ACL community saw fit to recognize a paper about an odd little ling construction.

We got an 🥂 Outstanding Paper Award!! Cannot be more grateful 🥹 This is super validating for our long pursuit of computational work on QUD. Congrats to the amazing @yatingwu.bsky.social, Ritika Mangla, Alex Dimakis, @gregdnlp.bsky.social

Wednesday at #EMNLP: @yatingwu.bsky.social will present our work connecting curiosity and questions in discourse. We built strong models to predict salience, outperforming large LLMs. 👉[Oral] Discourse+Phonology+Syntax2 10:30-12:00 @ Flagler also w/ Ritika Mangla @gregdnlp.bsky.social Alex Dimakis

Session 2 posters (11-12:30), Tuesday, Nov 12th Chantal Shaib & @jessyjli.bsky.social & @byron.bsky.social arxiv.org/abs/2407.00211 Learn more here - x.com/ChantalShaib...

I won't be at EMNLP, but come and see: 🔍 Detecting factual errors from LLMs (Liyan Tang) 🛠️ Detect, critique, & refine pipeline (Manya Wadhwa and Lucy Zhao) 🏭 Synthetic data generation (Abhishek Divekar) 📄 Fact-checking (Aniruddh Sriram) at FEVER t.co/fQbl0G7m23 (1st real post in the bluer skies!)

Our paper was accepted to EMNLP findings 😌 Can’t wait to talk about it at EMNLP and meet old friends and make new ones!

What differentiates in-group speech from out-group speech? I've been pondering this question for most of my PhD, and the final chapter of my dissertation tackles this question in a super interesting domain: comments from NFL🏈 team subreddits on live game threads. Our insight was to frame...🧵[1/7]

The 2nd Disinformation Day @ UT Austin is a 🔥event with experts from AI/NLP, Linguistics, Philosophy, and media; join us in this virtual event (for free)! Details & registration at disinfoday.github.io

We are pleased to announce that the first Conference on Language Modeling will be held at the University of Pennsylvania in Philadelphia at the Zellerbach Theatre. Thanks so much to UPenn CS as well as Mark Yatskar and Zachary Ives for facilitating the amazing venue.

As we move to subjective annotations and judgments in today’s LLM era, nuances can be expressed by asking human judges for an explanation. But how do we make use of these explanations in a principled way? Check out @manyawadhwa.bsky.social’s work 👇

Excited to share our updated preprint (w/ Jifan Chen, @jessyjli.bsky.social , @gregdnlp.bsky.social ) 📜 arxiv.org/pdf/2305.147... We show that LLMs can help understand nuances of annotation: they can convert the expressiveness of natural language explanations to a numerical form. 🧵

To appear #EMNLP2023! Can LMs simplify medical texts in non-English languages? We introduce⚕️MultiCochrane: the *first* multilingual, aligned dataset for this. arxiv.org/abs/2305.12532. Led by Sebastian Joseph, also w/ @byron.bsky.social Wei Xu