1/ 🌍 How does mixing data from hundreds of languages affect LLM training? In our new paper "Revisiting Multilingual Data Mixtures in Language Model Pretraining" we revisit core assumptions about multilinguality using 1.1B-3B models trained on up to 400 languages. 🧵👇

🌍Introducing BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data! LLMs learn from vastly more data than humans ever experience. BabyLM challenges this paradigm by focusing on developmentally plausible data We extend this effort to 45 new languages!

1/🚨 New preprint How do #LLMs’ inner features change as they train? Using #crosscoders + a new causal metric, we map when features appear, strengthen, or fade across checkpoints—opening a new lens on training dynamics beyond loss curves & benchmarks. #interpretability

🚨New Preprint! In multilingual models, the same meaning can take far more tokens in some languages, penalizing users of underrepresented languages with worse performance and higher API costs. Our Parity-aware BPE algorithm is a step toward addressing this issue: 🧵

Stop by our poster presentation at @iclr-conf.bsky.social and discuss real multilingual evaluation! Feel free to reach out anytime during the conference! We’d love to connect!

NEW PAPER ALERT: Generating visual narratives to illustrate textual stories remains an open challenge, due to the lack of knowledge to constrain faithful and self-consistent generations. Our #CVPR2025 paper proposes a new benchmark, VinaBench, to address this challenge.

Lots of great news out of the EPFL NLP lab these last few weeks. We'll be at @iclr-conf.bsky.social and @naaclmeeting.bsky.social in April / May to present some of our work in training dynamics, model representations, reasoning, and AI democratization. Come chat with us during the conference!

🚀 Introducing PICLe: a framework for in-context named-entity detection (NED) using pseudo-annotated demonstrations. 🎯 No human labeling needed—yet it outperforms few-shot learning with human annotations! #AI #NLProc #LLMs #ICL #NER

AI is reshaping #education, but are we ready? 🚨 Our new @pnas.org article explores how #LLMs challenge traditional assessments in higher education. Instead of banning #AI, we argue for redesigning assessments to emphasize real-world problem-solving and ethical AI use.

Excited to share our work on INCLUDE! 🚀 INCLUDE sets a new standard for #LLM benchmarks—spanning 44 languages with a focus on regional knowledge and cultural context 🌍 Time for LLMs to meet the world where it is, not where it’s translated to! #Multilingual #AI #NLProc

[email protected] is hiring for multiple positions in CS (including one open call): www.epfl.ch/about/workin... Apply to come join us in Beautiful Lausanne!

EPFL's new AI Center has a Call for applications for postdoc fellowships in all AI-related areas. Come join if you're interested in working with me and fantastic AI colleagues! Extra Perk: We actually do have lots of GPUs ! Deadline: November 29th More info at: www.epfl.ch/research/fun...