just had hechalou’s yin yang milk tea and i think i’ve transcended 🤤

during neurips, we kept the RL run going & model kept getting better 😂 Olmo 3.1 is a.. 🐡 32B Thinking, still best fully-open model to-date 🐠 32B Instruct, for ppl who hate long yapping, as good as qwen3 we added 10 more pages to the paper! thx for community feedback from convos at neurips

I'll be at #NeurIPS2025 from Tues-Sat! Come say hi 👋 if you wanna chat about 🦈 olmo 3 stories 🐟 pretraining data & evals 🍣 midtraining shouldnt exist 🐠 model specialization 🐡 AI for education 🍥 tabletop games

fml 🤦🏻‍♂️

we released Olmo 3! lot of exciting stuff but wanna focus on: 🐟Olmo 3 32B Base, the best fully-open base model to-date, near Qwen 2.5 & Gemma 3 on diverse evals 🐠Olmo 3 32B Think, first fully-open reasoning model approaching Qwen 3 levels 🐡12 training datasets corresp to different staged training

going live with a mukbang tmr 🍱

not happy abt gpt 5.1 update. it's making way more mistakes compared to gpt 5 on basic stuff latex table formatting errors (straight up missing "&" so columns misaligned, or dropping a whole column, or shifting values by 1 position), feels unusable imo 😒

picking between 3 checkpoints w/ same benchmark scores but what if one of them is agi

why intern at Ai2? 🐟interns own major parts of our model development, sometimes even leading whole projects 🐡we're committed to open science & actively help our interns publish their work reach out if u wanna build open language models together 🤝 links 👇

congrats to our olmo earth team 🌎 small multimodal foundation language models + system for finetuning for important uses like agriculture, wildfire management, conservation & more 🌿

woah guess VLMs for OCR the hottest research topic this week😆 since the first olmOCR, we've been.. 🔥training our VLM using RLVR with binary unit test rewards🔥 it's incredibly effective & unit test creation easy to scale w synthetic data pipelines check it out at olmocr.allen.ai

bye #colm2025 big fan of the montreal bagels 🥯 hot take I like them better than

come say hi at posters this morning for OLMo 2 and fluid benchmarking posters 👋 and dont miss @valentinhofmann.bsky.social's talk in morning #colm2025 @ai2.bsky.social vry proud of my gifs

@josephc.bsky.social @mariaa.bsky.social and I are at poster #21 findings from large scale survey of 800 researchers on how they use LMs in their research #colm2025

flyin to #colm2025 along w bunch of the @ai2.bsky.social team come chat w me about pretraining horror stories, data & evals, what we're cookin for next olmo, etc made a 🔥 poster for thursday sess, come say hi

5 am airport for the only direct flight from seattle to montreal #colm2025

not my project but I rlly like it working w cancer research center to analyze clinical data, but private data cant leave the center. so the team developed a tool that generates code for remote execution by the cancer center, developed on synthetic data, and now tested for realsies 🤩

had to explain to first time submitter why AC recommended accept ended up as reject 😮‍💨 been publishing long enough that i get why such things happen but can be rough

LM benchmark design requires 3 decisions, how to: 🐟 select test cases 🐠 score LM on each test 🦈 aggregate scores to estimate perf fluid benchmarking is simple: 🍣 find max informative test cases 🍥 estimate 'ability', not simple avg perf why care? turn ur grey noisy benchmarks to red ones!

scathing takedown of recent K2 Think model "evaluates on data it was trained on, relies on an external model and additional samples for its claimed performance gains, and artificially reduces the scores of compared models" www.sri.inf.ethz.ch/blog/k2think

COLM is coming up! Very excited. I'm starting to figure out two things: 1. A small invite-only dinner for Interconnects AI (Ai2 event news later). 2. Various research chats and catchups. Fill out the form below or email me if you're interested :) 🍁🇨🇦 Interest form: buff.ly/9nWBxZ9

🎙️ Say hello to OLMoASR—our fully open, from-scratch speech-to-text (STT) model. Trained on a curated audio-text set, it boosts zero-shot ASR and now powers STT in the Ai2 Playground. 👇

"Out of 13,048 reviewers..only 69 were deemed highly irresponsible..and enforcement was applied solely in those cases...These reviewers were contacted multiple times...as well as being personally contacted by the area chairs and senior area chairs, but still failed to fulfill them." 🫡🫡🫡

my favorite figure from work by @davidheineman.com if you're frustrated by LM evals, not knowing if results are real or noise, it's useful to decompose sources of variance: 🐠is there enough meaningful spread (signal) among compared models 🐟do scores vary between intermediate checkpoints (noise)

very nice work by @datologyai.com folks on synth data for pretraining very nice results over nemotron synth, which we've generally been impressed by rephrasing the web (arxiv.org/abs/2401.16380) seems very powerful & good demonstration of how to push it further

We’re releasing early pre-training checkpoints for OLMo-2-1B to help study how LLM capabilities emerge. They’re fine-grained snapshots intended for analysis, reproduction, and comparison. 🧵

huge thx to the NSF & NVIDIA for supporting our work on fully open AI model science & development 🤩

⚠️ AI-generated content may be inaccurate. Verify important information independently.

GPT-5 lands first place on NoCha, our long-context book understanding benchmark. That said, this is a tiny improvement (~1%) over o1-preview, which was released almost one year ago. Have long-context models hit a wall? Accuracy of human readers is >97%... Long way to go!

uniting the internet w chart crimes lol

thx to all the feedback from OSS community! our olmOCR lead Jake Poznanski shipped a new model fixing lotta issues + some more optimization for better throughput have fun converting PDFs!

olmoTrace for connecting model generations to training data won Best Paper for System Demonstrations at #ACL2025!

People at #ACL2025, come drop by our poster today & chat with me about how context matters for reliable language model evaluations! Jul 30, 11:00-12:30 at Hall 4X, board 424.

issues w preference LM benchmarks: 🐡data contains cases where the "bad" response is just as good as chosen one 🐟model rankings can feel off (claude ranks lower than expected) led by @cmalaviya.bsky.social, we study underspecified queries & detrimental effect on model evals; accepted to TACL 2025

lol 'what does skibidi mean' was not on my bingo card for @actinterp.bsky.social‬ workshop #icml2025 😆

presenting olmOCR at the poster session (2:15pm 211 West) for #codeml workshop at #icml2025! 🐟 fully open source OCR, comparable or better than frontier VLMs 🐠 all weights, data, code free & public 🐡 new benchmark of OCR "unit tests" on diverse PDFs & challenging OCR cases

Presenting two posters at ICML over the next two days: - Both at 11am - 1:30pm - Both about how to improve pre-training with domains - Both at stall # E-2600 in East Exhibition Hall A-B (!) Tomorrow: WebOrganizer w/ @soldaini.net & @kylelo.bsky.social Thursday: MeCo by @gaotianyu1350.bsky.social

will be at #icml2025, lemme kno if wanna chat about OLMo pretraining data curation, evaluation, data mixing, etc!👋 find us at poster sess on 📅Wed 7/16 @ 11am⏲️ to learn about Web Organizer, distilling web data taxonomies into small models & using them for LM data mixing!

only took few days to descend into madness

back from copenhagen & berkeley travels, now moving into new @ai2.bsky.social office!

🎧 youtu.be/YSFa_wOZPXg?...

thx for organizing! great to meet NLP folks & consume fancy bread 🥖🍞🥐

excited to release our new benchmark for OCR addressing 3 eval challenges: 🐟 coverage of many types of docs (born digital vs old scans, pages w tiny fonts, etc) 🐡 coverage of many different OCR targets (e.g. equations, tables, etc) 🐠 apples-to-apples comparison across systems

we won honorable mention for Best Paper at #CVPR2025 🏆 for Molmo & Pixmo, showing the value of high-quality data for VLMs! recalling when we released same time as Llama 3.2 😆 huge kudos to Matt Deitke, Chris Clark & Ani Kembhavi for their leadership on this project! @cvprconference.bsky.social

google down, guess ill go smell flowers or sthn 🤷‍♂️

excited to see this release of 1M public domain & CC zero books, digitized and OCR'd! 👏 big win for open data, congrats to the authors! arxiv.org/abs/2506.08300

fave part of this work is "ties" ⚖️ cases with many correct answers "Q: name a color of rainbow", a good reward model should know all answers "green", "blue", ... are correct but w/ no strong color preference congrats @saumyamalik.bsky.social et al @ai2.bsky.social & friends on RewardBench 2!

reviewers asking for comparison w qwen 3, have yall forgotten model was released a month ago lol 🤪

looks like same group got an AI generated paper accepted to ACL 😅 www.intology.ai/blog/zochi-acl

rlly love these graphs for contextualizing @colmweb.org review scores! 😍