Happy to have contributed to this

#MemoryModay #NLProc Countering Hateful and Offensive Speech Online - Open Challenges" by Plaza-Del-Arco, @debora_nozza, Guerini, Sorensen, Zampieri, 2024 is a tutorial on the challenges and solutions for detecting and mitigating hate speech.

#MemoryModay #NLProc Uma, A. N. et al. examine AI model training in 'Learning from Disagreement: A Survey'. Disagreement-handling methods' performance is shaped by evaluation methods & dataset traits.

#TBT #NLProc #MachineLearning #SafetyFirst 'Safety-Tuned LLaMAs: Improving LLMs Safety' by Bianchi et al. explores training LLMs for safe refusals, warns of over-tuning.

Come work with @deboranozza.bsky.social, me, and the lab in Milan!

We don't actually trust AI. We trust the companies behind it. As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.

We're hiring interns in the Computational Social Science group at Microsoft Research NYC! If you're interested in designing AI‑based systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026: apply.careers.microsoft.com/careers/job/...

After I shared “How to professor” last year, some people asked for a similar post on writing. Now I finally got around to typing up our lab's writing workshop slides. It covers basic advice for research papers and grant applications. Curious? Read it here: dirkhovy.com/post/2025_11...

#TBT #NLProc 'Respectful or Toxic?' by Plaza-del-Arco, @debora & @dirkhovy.bsky.social (2023) explores zero-shot learning for multilingual hate speech detection. Highlights prompt & model choice for accuracy. #AI #LanguageModels #HateSpeechDetection

#MemoryModay #NLProc 'Leveraging Social Interactions to Detect Misinformation on Social Media' by Fornaciari et al. (2023) uses combined text and network analysis to spot unreliable threads.

The Center for Information Technology Policy at Princeton invites applications for a Postdoctoral Fellow to work with Andy Guess (Politics/SPIA), Brandon Stewart (Sociology), and me (CS). puwebp.princeton.edu/AcadHire/app... Please apply before Sunday, the 13th of December!

#MemoryModay #NLProc 'Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models' by @paul-rottger.bsky.social et al. (2022). A suite of tests for 10 languages.

#TBT #NLProc 'Compromesso! Italian Many-Shot Jailbreaks Undermine LLM Safety' by Pernisi, @dirkhovy.bsky.social, @paul-rottger.bsky.social (2024). Paper highlights LLM vulnerability through Italian demos, more demos = more attack chances.

📢 Postdoc position 📢 I’m recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1. (Different from NYU Faculty Fellows, which are also great but less connected to my lab.) Link in 🧵

I will be @euripsconf.bsky.social this week to present our paper as non-archival at the PAIG workshop (Beyong Regulation: Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions! If you are at #EurIPS and want to chat about LLM's training data. Reach out!

Another exhausting day in the lab… conducting very rigorous panettone analysis. Pandoro was evaluated too, because we believe in fair experimental design.

#TBT #NLProc '@donyarn.bsky.social & @dirkhovy.bsky.social's 2024 paper, 'Conversations as a Source for Teaching Scientific Concepts' turns video dialogues into effective teaching tools.'

📢Submissions are open for EACL 2026 SRW **pre-submission mentorship**! Get valuable feedback on your research paper draft from experienced mentors before the deadline. Don't miss this chance to refine your work! 🔗 Submit here: openreview.net/group?id=eac... @eaclmeeting #NLProc

Here at UMass Amherst CICS, we’re searching for TT faculty in NLP – see the link from www.cics.umass.edu/about/employ... I’m happy to answer questions of course, too!

#MemoryModay #NLProc 'Hey Siri. Ok Google. Alexa: A topic modeling of user reviews for smart speakers,' by Nguyen & @dirkhovy.bsky.social decodes speaker reviews for user preferences using topic models. Domain knowledge needed for market analysis.

“Teacher Demonstrations in a BabyLM’s Zone of Proximal Development for Contingent Multi-Turn Interaction” selected for an Outstanding Paper Award at the BabyLM Challenge & Workshop!

What an inspiring week at #EMNLP2025 in Suzhou🇨🇳! Huge thanks to the organizers and everyone who stopped by our poster/talk!

See also @manoelhortaribeiro.bsky.social's post on this same topic: doomscrollingbabel.manoel.xyz/p/labeling-d...

Trying an experiment in good old-fashioned blogging about papers: dallascard.github.io/granular-mat...

#TBT #NLProc ' Attanasio et al. study asks 'Is It Worth the (Environmental) Cost?' analyzing continuous training for language models. Balances benefits, environmental impacts, for responsible use. #Sustainability'

#MemoryModay #NLProc ' 'State of Profanity Obfuscation in NLP Scientific Publications' probes bias in non-English papers. @deboranozza.bsky.social & @dirkhovy.bsky.social (2023) propose 'PrOf' to aid authors & improve access.

#TBT #NLProc Explore 'Wisdom of Instruction-Tuned LLM Crowds' by Plaza et al. LLM labels outperform single models in tasks & languages. But few-shot can't top zero-shot. Supervised models rule.

#MemoryModay #NLProc 'Universal Joy: A Data Set and Results for Classifying Emotions Across Languages' by Lamprinidis et al. (2021) explores how AI research affects our planet.

#TBT #NLProc "Explaining Speech Classification Models" by Pastor et al. (2024) makes speech classification more transparent! 🔍 Their research reveals which words matter most and how tone and background noise impact decisions.

#MemoryModay #NLProc 'Measuring Harmful Representations in Scandinavian Language Models' uncovers gender bias, challenging Scandinavia's equity image.

#TBT #NLProc Hessenthaler et al.'s 2022 work delves into AI's link with fairness & energy reduction in English NLP models, challenging bias reduction theories. #AI #sustainability

🎉 Congratulations to all #EMNLP2025 award winners 🎉 Starting with the ✨Best Paper award ✨: "Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index" by Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi aclanthology.org/2025.emnlp-m... 1/n

Maybe it is time to report *intra*-annotator agreement? aclanthology.org/2025.nlpersp...

Last week at @nlperspectives.bsky.social I presented work showing that annotators only provide the same label on ~75% of items across four NLP labelling tasks following a two week gap

You missed one: G. Abercrombie, T. Dinkar, A. Cercas Curry, V. Rieser & @dirkhovy.bsky.social Consistency is Key: Disentangling label variation in NLP with Intra-Annotator Agreement. @nlperspectives.bsky.social

Excited to head to Suzhou for the 30th edition of #EMNLP2025! 🎉 Had the great honor to serve as general chair this year. Looking forward to catching up with everyone and seeing some amazing #NLP research! 🤓📚

🗓️ Nov 5 – Main Conference Posters Personalization up to a Point 🧠 In the context of content moderation, we show that fully personalized models can perpetuate hate speech, and propose a policy-based method to impose legal boundaries. 📍 Hall C | 11:00–12:30

🗓️ Nov 5 – Main Conference Posters 📘 Biased Tales A dataset of 5k short LLM bedtime stories generated across sociocultural axes with an evaluation taxonomy for character-centric attributes and context-centric attributes. 📍 Hall C | 11:00–12:30

🗓️ Nov 5 - Demo Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification 🧩 Co-DETECT – an iterative, human-LLM collaboration framework for surfacing edge cases and refining annotation codebooks in text classification. 📍 Demo Session 2 – Hall C3 | 14:30–16:00

🗓️ Nov 6 – Findings Posters The “r” in “woman” stands for rights. 💬 We propose a taxonomy of social dynamics in implicit misogyny (EN,IT), auditing 9 LLMs — and they consistently fail. The more social knowledge a message requires, the worse they perform. 📍 Hall C | 12:30–13:30

🗓️ Nov 7 – Main Conference Posters Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance 🧍 Discussing different applications for LLM persona prompting, and how to measure their success. 📍 Hall C | 10:30–12:00

🗓️ Nov 7 – Main Conference Posters TrojanStego: Your Language Model Can Secretly Be a Steganographic Privacy-Leaking Agent 🔒 LLMs can be fine-tuned to leak secrets via token-based steganography! 📍 Hall C | 10:30–12:00

🗓️ Nov 8 – WiNLP Workshops No for Some, Yes for Others 🤖 We investigate how sociodemographic persona prompts affect false refusal behaviors in LLMs. Model and task type are the dominant factors driving these refusals.

🗓️ Nov 8 – NLPerspectives Workshops Balancing Quality and Variation 🧮 For datasets to represent diverse opinions, they must preserve variation while filtering out spam. We evaluate annotator filtering heuristics and show how they often remove genuine variation.

🗓️ Nov 8 – BabyLM Workshop Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction 👶 ContingentChat, a Teacher–Student framework that benchmarks and improves multi-turn contingency in a BabyLM trained on 100M words.

🗓️ Nov 8 – STARSEM Workshop Generalizability of Media Frames: Corpus Creation and Analysis Across Countries 📰 We investigate how well media frames generalize across different media landscapes. The 15 MFC frames remain broadly applicable, with minor revisions of the guidelines.

🗓️ Nov 6 – Oral Presentation (TACL) IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance ⚖️ A foundation for measuring LLM political bias in realistic user conversations. 📍 A303 | 10:30–12:00

Proud to present our #EMNLP2025 papers! Catch our team across Main, Findings, Workshops & Demos 👇

There’s plenty of evidence for political bias in LLMs, but very few evals reflect realistic LLM use cases — which is where bias actually matters. IssueBench, our attempt to fix this, is accepted at TACL, and I will be at #EMNLP2025 next week to talk about it! New results 🧵

Can LLMs learn to simulate individuals' judgments based on their demographics? Not quite! In our new paper, we found that LLMs do not learn information about demographics, but instead learn individual annotators' patterns based on unique combinations of attributes! 🧵