🇵🇭 One of my research interests is improving the state of Filipino NLP Happy to share that we're taking a major step towards this by introducing FilBench, an LLM benchmark for Filipino! Also accepted at EMNLP Main! 🎉 Learn more: huggingface.co/blog/filbench

Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚

I'll be at @aclmeeting.bsky.social‬ in Vienna! I'm going to present the ff first/co-first author works:

fun learning stuff (+ phew i haven't blogged in a long time!): ljvmiranda921.github.io/notebook/202...

We’re thrilled that SEA-VL has been accepted to the ACL 2025 (Main)! Thank you to everyone who contributed to this project 🥳 Paper: arxiv.org/abs/2503.07920 Project: seacrowd.github.io/seavl-launch/ #ACL2025NLP #SEACrowd #ForSEABySEA

We created Approximate Likelihood Matching, a principled (and very effective) method for *cross-tokenizer distillation*! With ALM, you can create ensembles of models from different families, convert existing subword-level models to byte-level and a bunch more🧵

🕵🏻💬 Introducing Feedback Forensics: a new tool to investigate pairwise preference data. Feedback data is notoriously difficult to interpret and has many known issues – our app aims to help! Try it at app.feedbackforensics.com Three example use-cases 👇🧵

OLMo 2 0325 32B Preference Mixture: Solves AI alignment challenges through diverse preferences - Combines 7 datasets - Filters for instruction-following capability - Balances on-policy and off-policy prompts - Enabled successful DPO of OLMo-2-0325-32B model huggingface.co/datasets/all...

Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! It demonstrates that our recipe, which includes RVLR scales to 405B - with performance on par with GPT-4o, & surpassing prior open-weight post-trained models of the same size including Llama 3.1.

kicking off 2025 with our OLMo 2 tech report while payin homage to the sequelest of sequels 🫡 🚗 2 OLMo 2 Furious 🔥 is everythin we learned since OLMo 1, with deep dives into: 🚖 stable pretrain recipe 🚔 lr anneal 🤝 data curricula 🤝 soups 🚘 tulu post-train recipe 🚜 compute infra setup 👇🧵

BERT is BACK! I joined a collaboration with AnswerAI and LightOn to bring you the next iteration of BERT. Introducing ModernBERT: 16x larger sequence length, better downstream performance (classification, retrieval), the fastest & most memory efficient encoder on the market. 🧵

New research reveals a worrying trend: AI's data practices risk concentrating power overwhelmingly in the hands of dominant technology companies. With analysis from @shaynelongpre.bsky.social @sarahooker.bsky.social @smw.bsky.social @giadapistilli.com www.technologyreview.com/2024/12/18/1...

Stop by our #NeurIPS tutorial on Experimental Design & Analysis for AI Researchers! 📊 neurips.cc/virtual/2024/tutorial/99528 Are you an AI researcher interested in comparing models/methods? Then your conclusions rely on well-designed experiments. We'll cover best practices + case studies. 👇

We just updated the AI for Humanists guide to model selection to include Llama 3.3, and a recommended best cost/capability tradeoff, llama 3.1 8B. What have you tried, and what would you suggest? aiforhumanists.com/guides/models/

the science of LMs should be fully open✨ today @akshitab.bsky.social @natolambert.bsky.social and I are giving our #neurips2024 tutorial on language model development. everything from data, training, adaptation. published or not, no secrets 🫡 tues, 12/10, 9:30am PT ☕️ neurips.cc/virtual/2024...

Come chat with me at #NeurIPS2024 and learn about how to use Paloma to evaluate perplexity over hundreds of domains! ✨We have stickers too✨

⭐️ We're going to launch Grassroots Science, a year-long ambitious, massive-scale, fully open-source initiative aimed at developing multilingual LLMs aligned to diverse and inclusive human preferences in Feb 2025. 🌐 Check our website: grassroots.science. #NLProc #GrassrootsScience

We're releasing the largest Universal Dependencies (UD) treebank for Tagalog, UD-NewsCrawl! This dataset has been a long time coming, but glad to see this through: 15k+ sentences versus the previous ~150 sents from older Tagalog treebanks. 🤗 : huggingface.co/datasets/UD-... 📝 : Paper soon!

I am seriously behind uploading Learning Machines videos, but I did want to get @jonathanberant.bsky.social's out sooner than later. It's not only a great talk, it also gives a remarkably broad overview and contextualization, so it's an excellent way to ramp up on post-training youtu.be/2AthqCX3h8U

My favorite part about this release is that we were able to replicate our findings from the Tülu 3 post-training recipe here (e.g., on-policy preferences, RLVR) and found significant performance gains in our -DPO and -Instruct models! Find all artifacts here: huggingface.co/collections/...

Happy to be part of Tülu 3! Great effort to make the post-training stage open-source 😄 I worked on scaling our synthetic preference data (around 300k preference pairs for 70B) that led to performance gains when trained on using DPO.

Meet Tülu 3, a set of state-of-the-art instruct models with fully open data, eval code, and training algorithms. We invented new methods for fine-tuning language models with RL and built upon best practices to scale synthetic instruction and preference data. Demo, GitHub, paper, and models 👇

I've spent the last two years scouring all available resources on RLHF specifically and post training broadly. Today, with the help of a totally cracked team, we bring you the fruits of that labor — Tülu 3, an entirely open frontier model post training recipe. We beat Llama 3.1 Instruct. Thread.