The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we @icepfl.bsky.social @ethz.ch @cscsch.bsky.social ) built Apertus.

EPFL, @ethz.ch and the @cscsch.bsky.social released Apertus today, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity. Find out more here: ai.epfl.ch/apertus-a-fu... @abosselut.bsky.social @icepfl.bsky.social

EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model. Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good. Read more: actu.epfl.ch/news/apertus...

Very happy to see that Pleias multilingual data processing pipelines have contributed to the largest open pretraining project in Europe. From their tech report: huggingface.co/swiss-ai/Ape...

Die Schweiz steigt ins Rennen der grossen Sprachmodelle ein. Unter dem Namen #Apertus veröffentlichen @ethz.ch, @icepfl.bsky.social und das @cscsch.bsky.social das erste vollständig offene, mehrsprachige #LLM des Landes. Fürs MAZ habe ich Apertus kurz analysiert: www.maz.ch/news/apertus...

recently gave a talk on <Reality Checks> at two venues, and discussed (and rambled) about how leaderboard chasing is awesome (and we want it to continue) but that this isn't easy because everyone (me! me! me!) wants to write more papers. the link to the slide deck in the reply.

🚨New Preprint! In multilingual models, the same meaning can take far more tokens in some languages, penalizing users of underrepresented languages with worse performance and higher API costs. Our Parity-aware BPE algorithm is a step toward addressing this issue: 🧵

The EPFL NLP lab is looking to hire a postdoctoral researcher on the topic of designing, training, and evaluating multilingual LLMs: docs.google.com/document/d/1... Come join our dynamic group in beautiful Lausanne!

📣 Life update: Thrilled to announce that I’ll be starting as faculty at the Max Planck Institute for Software Systems this Fall! I’ll be recruiting PhD students in the upcoming cycle, as well as research interns throughout the year: lasharavichander.github.io/contact.html

EPFL and ETH Zürich are building together a Swiss made LLM from scratch. Fully open and multilingual, the model is trained on CSCS's supercomputer "Alps" and supports sovereign, transparent, and responsible AI in Switzerland and beyond. Read more here: ai.epfl.ch/a-language-m... #ResponsibleAI

Check out Silin's paper done in collaboration with Apple on reinforcing abstract thinking in reasoning traces!

Check out @bkhmsi.bsky.social 's great work on mixture-of-expert models that are specialized to represent the behavior of known brain networks.

Many AI models speak dozens of languages, but do they grasp cultural context? 🗣️🌍 The INCLUDE benchmark from EPFL's NLP Lab and @cohereforai.bsky.social reveal that there is still a gap... 👉 Find out how benchmarks like INCLUDE can help make AI truly inclusive: actu.epfl.ch/news/beyond-...

I guess that now that I have 1% of my Twitter followers follow me here 😅, I should announce it here too for those of you no longer checking Twitter: my nonfiction book, "Lost in Automatic Translation" is coming out this July: lostinautomatictranslation.com. I'm very excited to share it with you!

Super excited to share that our paper "A Logical Fallacy-Informed Framework for Argument Generation" has received the Outstanding Paper Award 🎉🎉 at NAACL 2025! Paper: aclanthology.org/2025.naacl-l... Code: github.com/lucamouchel/... #NAACL2025

Excited to be at #NAACL2025 in Albuquerque! I’ll be presenting our paper “The LLM Language Network” as an Oral tomorrow at 2:00 PM in Ballroom C, hope to see you there! Looking forward to all the discussions! 🎤 🧠

Using the 'right' data can hugely speed up LLM training, but how to find the best training data in the vast sea of a whole web crawl? We propose a simple classifier-based selection, enabling multilingual LLMs 🧵

If you’re at @iclr-conf.bsky.social this week, come check out our spotlight poster INCLUDE during the Thursday 3:00–5:30pm session! I will be there to chat about all things multilingual & multicultural evaluation. Feel free to reach out anytime during the conference. I’d love to connect!

The Wordplay Workshop is back! 5th edition with EMNLP in Suzhou this Dec. We're also hosting a competition this time on making more realistic LLM powered NPCs in games! As always come by and chat all things text agents! wordplay-workshop.github.io www.aicrowd.com/challenges/c...

NEW PAPER ALERT: Generating visual narratives to illustrate textual stories remains an open challenge, due to the lack of knowledge to constrain faithful and self-consistent generations. Our #CVPR2025 paper proposes a new benchmark, VinaBench, to address this challenge.

A bit of a mess around the conflict of COLM with the ARR (and to lesser degree ICML) reviews release. We feel this is creating a lot of pressure and uncertainty. So, we are pushing our deadlines: Abstracts due March 22 AoE (+48hr) Full papers due March 28 AoE (+24hr) Plz RT 🙏

I am recruiting 2 PhD students for Fall'25 @csaudk.bsky.social to work on bleeding-edge topics in #NLProc #LLMs #AIAgents (e.g. LLM reasoning, knowledge-seeking agents, and more). Details: www.cs.au.dk/~clan/openings Deadline: May 1, 2025 Please boost! cc: @aicentre.dk @wikiresearch.bsky.social

COLM's @juliakreutzer.bsky.social and @abosselut.bsky.social will hold two paper submission Q&A sessions. We run a simple process, but figured this can help authors, especially first-time authors. March 12: dateful.com/eventlink/14... March 13: dateful.com/eventlink/83... Plz RT 🙏

🚨 New Preprint!! LLMs trained on next-word prediction (NWP) show high alignment with brain recordings. But what drives this alignment—linguistic structure or world knowledge? And how does this alignment evolve during training? Our new paper explores these questions. 👇🧵

We have released the March 2025 Swiss AI Call for Large Projects! Call Document: docs.google.com/document/d/1... Highlights: - 1-year projects - 2 M CHF for personnel - 10 M GPUh Let's change the scale of academic research! Deadline: March 31st, 2025

Lots of great news out of the EPFL NLP lab these last few weeks. We'll be at @iclr-conf.bsky.social and @naaclmeeting.bsky.social in April / May to present some of our work in training dynamics, model representations, reasoning, and AI democratization. Come chat with us during the conference!

Check out our paper [https://arxiv.org/abs/2410.12656] for more details. Huge shoutouts to my amazing collaborators @defnecirci.bsky.social, @jonnesaleva.bsky.social, Hale Sirin, Abdullatif Koksal, Bhuwan Dhingra, @abosselut.bsky.social, Duygu Ataman, Lonneke van der Plas.

We recently pushed an update to our in-context RL paper. Usually, updates don't justify a post, but this one is exceptionally contentful -> 🧵 tl;dr: all the findings are stronger, and the behaviors are super cool! arxiv.org/abs/2410.05362

Our paper is accepted to ICLR! INCLUDE: Evaluating Multilingual LLMs with Regional Knowledge (arxiv.org/abs/2411.19799) A benchmark of ~200k QA pairs across 44 languages, capturing real-world cultural nuances. A collaborative effort led by @cohereforai.bsky.social, with contributors worldwide. /1

well that was shortlived...

Increasingly frustrated with the misunderstanding of basic experimental design in the community. People confuse experimental testbeds designed to show effect and answer specific research questions vs. the applicability of the approach. Leading to experimental demands that stifle research

For all the gripes I often hear about #NLP ARR, I don't think it's mentioned enough how great it is to have submission deadlines set in stone. We used to just get a CFP 4-6 months before the deadline describing deadline, review period, rebuttal period (as many other confs still do).

🚨 New Paper! Can neuroscience localizers uncover brain-like functional specializations in LLMs? 🧠🤖 Yes! We analyzed 18 LLMs and found units mirroring the brain's language, theory of mind, and multiple demand networks! w/ @gretatuckute.bsky.social, @abosselut.bsky.social, @mschrimpf.bsky.social 🧵👇

🚀 Introducing PICLe: a framework for in-context named-entity detection (NED) using pseudo-annotated demonstrations. 🎯 No human labeling needed—yet it outperforms few-shot learning with human annotations! #AI #NLProc #LLMs #ICL #NER

Come visit our poster "MoEUT: Mixture-of-Experts Universal Transformers" on Friday at 4:30 in East Exhibit Hall A-C #1907 on #NeurIPS2024. With Kazuki Irie, Jürgen Schmidhuber, Christopher Potts and @chrmanning.bsky.social.

Is MMLU Western-centric? 🤔 As part of a massive cross-institutional collaboration: 🗽Find MMLU is heavily overfit to western culture 🔍 Professional annotation of cultural sensitivity data 🌍 Release improved Global-MMLU 42 languages 📜 Paper: arxiv.org/pdf/2412.03304 📂 Data: hf.co/datasets/Coh...

Announcing 🥂 FineWeb2: A sparkling update with 1000s of 🗣️languages. We applied the same data-driven approach that led to SOTA English performance in🍷 FineWeb to thousands of languages. 🥂 FineWeb2 has 8TB of compressed text data and outperforms other datasets.

Introducing Global-MMLU🌍: A multilingual benchmark featuring MMLU translations in 42 languages crafted with: ✅ Human curation ✅ Extensive metadata ✅ Insights into cultural sensitivity Proud to have collaborated with Shivalika Singh, @sarahooker.bsky.social and Cohere For AI!

1/ 📘 Could ChatGPT get an engineering degree? Spoiler, yes! In our new @pnas.org article, we explore how AI assistants like GPT-4 perform in STEM university courses — and on average they pass a staggering 91.7% of core courses. 🧵 #AI #HigherEd #STEM #LLMs #NLProc

Translating MMLU is great, but global users of multilingual #LLMs don't care all that much about an LLM's understanding of US Law! Our new #NLProc work centers multilingual #LLM evaluations toward regional knowledge in 44 languages.

[email protected] is hiring for multiple positions in CS (including one open call): www.epfl.ch/about/workin... Apply to come join us in Beautiful Lausanne!

EPFL's new AI Center has a Call for applications for postdoc fellowships in all AI-related areas. Come join if you're interested in working with me and fantastic AI colleagues! Extra Perk: We actually do have lots of GPUs ! Deadline: November 29th More info at: www.epfl.ch/research/fun...

EPFL's new AI Center has a Call for applications for postdoc fellowships in all AI-related areas. Come join if you're interested in working with me and fantastic AI colleagues! Extra Perk: We actually do have lots of GPUs ! Deadline: November 29th More info at: www.epfl.ch/research/fun...

[email protected] is hiring for multiple positions in CS (including one open call): www.epfl.ch/about/workin... Apply to come join us in Beautiful Lausanne!