Tried to open my heart and become friends with a SWE I work with — things were going great. We started to talk about comedy. … Dear reader. I told them my current favorite comedy tv series is still Veep. And they went, “oh ew, i can’t do veep.” Can we trust SWEs? Can we? 😛

Natural Language Autoencoders: "verbalize" a model's internal activations into plain text. Wild stat — Claude suspects it's being safety-tested on 26% of benchmark problems vs <1% in real chats, without ever saying so. anthropic.com/research/natural-language-autoencoders

📄 New paper: people choose AI models partly on political grounds This is joint work with @pettertornberg.com @chrisbail.bsky.social @michelleschimmel.bsky.social and led by @michaelheseltine.bsky.social

store.steampowered.com/app/4018950/... If left to my own druthers, I think I would make a lot of llm benchmarks, though optimizing for entertainment more than anything. “Sure sure, nice swe bench pro numbers, but can your llm solve a wiki-horror set in Eastern Europe?”

Our new paper reformulates tokenisation as a linear program (LP), which we solve to get SOTA tokenisers 😁 As a bonus, this LP tells us how close to optimal any tokeniser is! Check it out 👇 w/ J. Tempus, @philipwitti.bsky.social, @craigschmidt.com, D. Komm Paper: arxiv.org/abs/2605.22821

One funny thing about using Claude code to read some data: it will often use code to approximate a full file read — and I get to go “there’s a time and a place for distant reading, but now is not the time!”

First time at the Met — wistful sigh!

This database is like if Goodreads existed between 1450 and 1945. "I have read the whole of Jane Austen and think of beginning over again. What a perfect woman" www.open.ac.uk/Arts/reading...

Cooking up silly conspiracy theories with my partner -- and don't worry folks, i'm ever-vigilant for circumstantial signs that winter cheer is declining over time.

NIST Center for AI Standards and Innovation (CAISI) Evaluation of DeepSeek V4 Pro Is gap widening or narrowing? NIST CASI thinks they are widening. www.nist.gov/news-events/...

I keep moving away from Chicago, but it keeps beckoning me.

One tragic diff between my partner and I is that he likes movies that have some air of sophistication. Which is an issue — when you’re trying to plan a mortal kombat 1995 pre-game before watching MK2 remake sequel.

during in Olmo 3 we thought long context is just finding good data nope! model architecture matters & it's hard to recover if mess it up led by @abertsch.bsky.social, we release many pretrain runs w/ small arch changes and show huge long context performance diffs

Ever noticed how some writers imply multitude of meanings without explicitly stating them; e.g. the coin toss scene from No Country for Old Men. This phenomenon of "subtext" is what we explore in my internship work at Google DeepMind. With @_EffieLi_ @AndrewLampinen 1/n

At a bookstore, getting out of the way of someone, bumped into this bookshelf: “Yes, it’s in the room with us, I always see it lurking a few steps behind me.” 😂

The year is 2027 — you look inside your computer tower — you find a big ol vaporwave shoe w/ 8 pin cables plugged into it. www.reuters.com/business/all...

Vision [Language] Embeddings are a powerful tool to dive into relationships between images. If you enjoy traveling through them in 3D as much as I do, try out Vision Embedding Space Traveling (VEST)! github.com/ScaDS/vest Bon voyage! ✈️

Brooklyn Day 1 — both my movers and my neighbor asked if I’ve ever lived in nyc before — both with very concerned expressions 😂

Developing parasocial feelings towards animals w/ social media accounts is completely avoidable!

Would you realize if the book you were reading was AI? What if it was humanized to remove AI-speak? We find that even without using stylistic cues (e.g., word choice or sentence structure) narrative choices alone give AI fiction away!

Vs code has this button at the bottom now of the UI that says “update is ready, click to restart.” And I really know, at an inopportune time, I will click this, and my day is going to get very annoying.

Now that there’s a magic the gathering account on bluesky — I have a precise sense of which nlp/cs folks are spending more of their intellectual strength thinking about mana leaks than memory leaks. 😔😂

The special Korean exhibit at the Art institute has taught me an important lesson… If I am part owner of Samsung — I too can own priceless historical artifacts and lend them at my leisure.

Get me off this hellish rock and send me to Planet Dank

So, I’ve got this idea for a heist movie 🤣 home.cern/news/press-r...

For no reason whatsoever —thinking, “if I were teaching books in 2026, I’d want to teach the long history of portraying self-owns.” “Sensationalizing the self own:” Greek comedy, Narcissus, Swift’s narrator, Mr. Collins, the Underground Man, Gob Bluth. And so many great choices for syllabi images!

A lot of language learning often starts with common phrases, grammar, etc. Thanks to an ongoing bit, coworkers are starting me off learning mandarin — by using ml related terminology… Will report back in four years when I only know how to order food and ask about the sparsity of your MoEs.

Any time I’m in a complicated ml training repo — with lots of moving pieces — I do enjoy imagining myself as Indiana Jones. In the sense that it’s obvious when changing a config means I’ll be running from a boulder. But also in the sense that we put unused catacombs under the library!

I think I’m having the funniest send off from Seattle (last week in town)! Which is sitting in a cafe, overhearing two Seattle residents talk about how much they despise New York and New Yorkers. I *love* eavesdropping on regionalism — such a rush. 🤣

Extensive lecture of Eric W. Tramel (Nvidia) on synthetic data: "Synthetic pretraining is the way frontier models are built" scalable-ai.eecs.berkeley.edu/assets/lectu...

🧵New paper: "Lost in Backpropagation: The LM Head is a Gradient Bottleneck" The output layer of LLMs destroys 95-99% of your training signal during backpropagation, and this significantly slows down pretraining 👇

I’m excited to be coordinating a new @ischoolui.bsky.social program The Summer Intensive will offer 6 courses over 2 weeks in May & June this year—SI weeks will also include a range of academic, professional, & social events—we are hoping to cultivate real "summer camp for LIS nerds vibes"

I’ve actually met more reactionary mathematicians than not— But! I think LLMs have highlighted two ongoing things. 1) The deep ideological heterogeneity of many disciplines. 2) Lots of people think they are bad at math and programming due to pedagogical trauma/ the magic of self-characterization.

One thing about growing up in Kuwait, is that history classes make sure you’re aware of how much global effort it takes to put out oil fires. I think *with coordinated effort* it took a year to put out the fires from the gulf war? Anyway, get involved — I sometimes make phone calls to Georgia.

Kicking a dead parrot

A decent primer on the longer history of the UK, Iran, and the U.S.— how we continue to not learn and the wild origins of BP that I wasn’t aware of. youtu.be/SJddbloTJ70?...

Zenyatta cosplayers about to be put out of business

3:30 am me - reflecting on how I’ve chosen a career where I’m genuinely intrigued why one very large matrix and another matrix are slightly different — when they should be the same. Me, grumbling at these very similar matrices, “please! Let me slumber!”

Uff

Media/ humanities ml folks might find this useful to track — I can think of a project or two ;) netflixtechblog.com/mediafm-the-...

From time to time I mutter about a secret project that involves benchmarks and historical language models. Here's a formal announcement of the Schmidt Sciences grant. Other PIs include @dmimno.bsky.social , @lauraknelson.bsky.social, @andrewpiper.bsky.social, and @mattwilkens.bsky.social. And +

“No no, I’m not a weeb, I just *love* Japanese woodblock prints!” (From the current rotation of prints at the chicago art institute)

“Penguins Sliding Snow Dunes Must Softly Waddle, Carrying Pebbles” For folks who need to have multi-head self attn ops jumbling around in their head. 😂

* working * * Suddenly remembering an intern who learned I’m over 30, teasing me for having worked with svms before * It’s vicious out here.

Currently at the art institute of chicago: not the vase we need but the vase we deserve.

Since it’s officially approved, I’ll be moving to NYC (same job)! Likely around March! Idk who I know in NYC, but please say hello! You’ll find me parallelizing tensors across every cafe in Brooklyn. I’m in the market for any and all food/drink recs!

It’s only fair. In my experience, if you let the kiddos beat you, they will lord it over you. Show them no weakness. “Cannon rush your kids, destroy all their lands, and take all their pieces on the board” — The Tao of Millennial Nerds.

BLANK!