1/ Are two #LLMs better than one for equitable cultural alignment? 🌍 We introduce a Multi-Agent Debate framework — where two LLM agents debate the cultural adaptability of a given scenario. #ACL2025 🧵👇

Trying to collect all the MT people here. I probably missed many. Ping me! bsky.app/starter-pack...

1/ How can a monolingual English speaker 🇺🇸 decide if an automatic French translation 🇫🇷 is good enough to be shared? Introducing ❓AskQE❓, an #LLM-based Question Generation + Answering framework that detects critical MT errors and provides actionable feedback 🗣️ #ACL2025

How does the public conceptualize AI? Rather than self-reported measures, we use metaphors to understand the nuance and complexity of people’s mental models. In our #FAccT2025 paper, we analyzed 12,000 metaphors collected over 12 months to track shifts in public perceptions.

Multilinguality is happening at #NAACL2025 @crystinaz.bsky.social @oxxoskeets.bsky.social @dayeonki.bsky.social @onadegibert.bsky.social

Starting my journey on Bluesky with a topic that I care deeply about: AI tools can support creators in various ways, but disclosing AI use may risk devaluing creative work. Check out our abstract here: angelhwang.github.io/doc/ic2s2_AI... Inspired by our past work: arxiv.org/abs/2411.13032

🚨 New Paper 🚨 1/ We often assume that well-written text is easier to translate ✏️ But can #LLMs automatically rewrite inputs to improve machine translation? 🌍 Here’s what we found 🧵

🚨 NEW WORKSHOP ALERT 🚨 We're thrilled to announce the first-ever Tokenization Workshop (TokShop) at #ICML2025 @icmlconf.bsky.social! 🎉 Submissions are open for work on tokenization across all areas of machine learning. 📅 Submission deadline: May 30, 2025 🔗 tokenization-workshop.github.io

Thrilled our global data ecosystem audit was accepted to #ICLR2025! Empirically, it shows: 1️⃣ Soaring synthetic text data: ~10M tokens (pre-2018) to 100B+ (2024). 2️⃣ YouTube is now 70%+ of speech/video data but could block third-party collection. 3️⃣ <0.2% of data from Africa/South America. 1/

How do LLMs compare to human crowdworkers in annotating text spans? 🧑🤖 And how can span annotation help us with evaluating texts? Find out in our new paper: llm-span-annotators.github.io Arxiv: arxiv.org/abs/2504.08697

Call for participation: We just opened the registration for this year's MT Marathon in August in Helsinki, Finland: blogs.helsinki.fi/language-tec..., featuring: - Ayodele Awokoya - Wilker Aziz - Marta Costa-Jussa - Barry Haddow - Amit Moryosse - Sara Papi - Jörg Tiedemann - Marco Turchi

Come to Helsinki for the 18th MT Marathon! Sponsored by EAMT @ufal-cuni.bsky.social

** New parallel data set ** . We've just released HPLT v2.0, a parallel data set of 50 languages paired with English, 380M sentence pairs in total. Extracted from the Internet Archive and Common Crawl hplt-project.org/datasets/v2.0

Brilliant and necessary work by Pombal et al. about metric interference in MT system development and evaluation: arxiv.org/abs/2503.08327 Are we developing better systems or are we just gaming the metrics? And how do we address this? Super (m)interesting! 👀

Introducing 🐻 BEARCUBS 🐻, a “small but mighty” dataset of 111 QA pairs designed to assess computer-using web agents in multimodal interactions on the live web! ✅ Humans achieve 85% accuracy ❌ OpenAI Operator: 24% ❌ Anthropic Computer Use: 14% ❌ Convergence AI Proxy: 13%

New preprint w/ @jennhu.bsky.social @kmahowald.bsky.social : Can LLMs introspect about their knowledge of language? Across models and domains, we did not find evidence that LLMs have privileged access to their own predictions. 🧵(1/8)

OK, every year I try to explain to my students how LLMs work, and every year I have to do a big trawl for good resources and activities. Here's this year's haul of *introductory* materials. (In-class activities + visualizations, not so much readings.)

Big news from WMT! 🎉 We are expanding beyond MT and launching a new multilingual instruction shared task. Our goal is to foster truly multilingual LLM evaluation and best practices in automatic and human evaluation. Join us and build the winning multilingual system! www2.statmt.org/wmt25/multil...

self-insert, but if you are looking for something multilingual and public domain, we have PoeTree: a collection of poetry corpora with Python & R access points (can get data directly into your jupyter notebook) : versologie.cz/poetree/

Lots of work coming soon to @iclr-conf.bsky.social and @naaclmeeting.bsky.social in April/May! Come chat with us about new methods for interpreting and editing LLMs, multilingual concept representations, sentence processing mechanisms, and arithmetic reasoning. 🧵

🚨 Our team at UMD is looking for participants to study how #LLM agent plans can help you answer complex questions 💰 $1 per question 🏆 Top-3 fastest + most accurate win $50 ⏳ Questions take ~3 min => $20/hr+ Click here to sign up (please join, reposts appreciated 🙏): preferences.umiacs.umd.edu

Happy to say that our paper "Beyond Literal Token Overlap: Token Alignability for Multilinguality" will be presented at #NAACL2025! This is work with @tomlim.bsky.social, @jlibovicky.bsky.social, and Alex Fraser. arxiv.org/abs/2502.06468 #newpaper #NLP #NLProc

✨New pre-print✨ Crosslingual transfer allows models to leverage their representations for one language to improve performance on another language. We characterize the acquisition of shared representations in order to better understand how and when crosslingual transfer happens.

This is a really neat use case for AI—checking whether claims are actually supported by the given citations.

📢New Paper Alert!🚀 Human alignment balances social expectations, economic incentives, and legal frameworks. What if LLM alignment worked the same way?🤔 Our latest work explores how social, economic, and contractual alignment can address incomplete contracts in LLM alignment🧵

We hope Aya enables researchers and developers throughout the world to build upon this technology, ask deeper questions about multilingual AI, and develop tools that can support their communities. Learn more: cohere.com/blog/aya-vis...

✨👓 Aya Vision is here 👓✨ A multilingual, multimodal model designed to understand across languages and modalities (text, images, etc) to bridge the language gap and empower global users!

Huge shoutout to colleagues at Google & Unbabel for extending our WMT24 testset to 55 languages in four domains, this is game changer! 🚀 I really hope it puts the final nail in the coffin of FLORES or WMT14. The field is evolving, legacy testsets can't show your progress arxiv.org/abs/2502.124...

Induction heads are commonly associated with in-context learning, but are they the primary driver of ICL at scale? We find that recently discovered "function vector" heads, which encode the ICL task, are the actual primary mechanisms behind few-shot ICL! arxiv.org/abs/2502.14010 🧵👇

People are increasingly relying on AI assistance, but *how* they use AI advice is influenced by their trust in the AI, which the AI is typically blind to. What if they weren’t? We show that adapting AI assistants' behavior to user trust mitigates under- and over-reliance! arxiv.org/abs/2502.13321

There's been a lot of work on "culture" in NLP, but not much agreement on what it is. A position paper by me, @dbamman.bsky.social, and @ibleaman.bsky.social on cultural NLP: what we want, what we have, and how sociocultural linguistics can clarify things. Website: naitian.org/culture-not-... 1/n

Thrilled to be at #AAAI2025 for our tutorial, “AI Data Transparency: The Past, Present, and Beyond.” We’re presenting the state of transparency, tooling, and policy, from the Foundation Model Transparency Index, Factsheets, the the EU AI Act to new frameworks like @MLCommons’ Croissant. 1/

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

⚠️Current methods for generating instruction-following data fall short for long-range reasoning tasks like narrative claim verification. We present CLIPPER ✂️, a compression-based pipeline that produces grounded instructions for ~$0.5 each, 34x cheaper than human annotations.

🚨 New Position Paper 🚨 Multiple choice evals for LLMs are simple and popular, but we know they are awful 😬 We complain they're full of errors, saturated, and test nothing meaningful, so why do we still use them? 🫠 Here's why MCQA evals are broken, and how to fix them 🧵

Asking the right questions can make or break decisions in fields like medicine, law, and beyond✴️ Our new framework ALFA—ALignment with Fine-grained Attributes—teaches LLMs to PROACTIVE seek information through better questions through **structured rewards**🏥❓ (co-led with @jiminmun.bsky.social) 👉🏻🧵

starter pack for the Computational Linguistics and Information Processing group at the University of Maryland - get all your NLP and data science here! go.bsky.app/V9qWjEi

Aya Expanse technical report is here and I couldn’t be prouder! 🎉 Check out the report: arxiv.org/abs/2412.04261

I noticed a lot of starter packs skewed towards faculty/industry, so I made one of just NLP & ML students: go.bsky.app/vju2ux Students do different research, go on the job market, and recruit other students. Ping me and I'll add you!

Is MMLU Western-centric? 🤔 As part of a massive cross-institutional collaboration: 🗽Find MMLU is heavily overfit to western culture 🔍 Professional annotation of cultural sensitivity data 🌍 Release improved Global-MMLU 42 languages 📜 Paper: arxiv.org/pdf/2412.03304 📂 Data: hf.co/datasets/Coh...