🌊 Global Mangrove Watch is using OlmoEarth to refresh mangrove map baselines faster, with higher accuracy & less manual annotation—allowing orgs + governments to respond to threats more quickly. Learn more → buff.ly/6xLHLk6

ml halloween costume concept

Yes!! A POMDP world model benchmark with controlled test environments. So excited to play with this

Really proud of this project led by @kylecranmer.bsky.social and created alongside these awesome collaborators and friends. I believe this is important work that will help researchers make grounded decisions when building multimodal models with many diverse inputs. Add it to your paper pile!

Cornell (NYC and Ithaca) is recruiting AI postdocs, apply by Nov 20, 2025! If you're interested in working with me on technical approaches to responsible AI (e.g., personalization, fairness), please email me. academicjobsonline.org/ajo/jobs/30971

Here is the recipe from the latest Thinking Machines blogpost about late post-training: - generate student rollouts - query teacher distribution forced on student history - update using the reverse KL divergence at each step thinkingmachines.ai/blog/on-poli...

Did an intro to tokenization lecture today and worked in this thread.

The first fantastic paper on scaling RL with LLMs just dropped. I strongly recommend taking a look and will be sharing more thoughts on the blog soon. The Art of Scaling Reinforcement Learning Compute for LLMs Khatri & Madaan et al. buff.ly/olKwF3X

NVIDIA sent me preview hardware of their new DGX Spark 128GB ARM64 4TB "AI supercomputer" - it's a very neat little device, here are my notes so far simonwillison.net/2025/Oct/14/...

Multi-Head Latent Attention 🔗 github.com/rasbt/LLMs-f...

⚠️ You have marked yourself as an untrusted node in the epistemic network

improving pretrained LLMs by searching over iid-noised params, using a reward score (aka fitness criterion) for weight-merging

We are excited to announce 4 outstanding papers 🏆🏆🏆🏆 --> 🧵

LLMs are currently this one big parameter block that stores all sort of facts. In our new preprint, we add context-specific memory parameters to the model, and pretrain the model along with a big bank of memories. 📑 arxiv.org/abs/2510.02375 [1/10]🧵

accepted papers, COLM 2025 colmweb.org/AcceptedPape...

Paper: arxiv.org/pdf/2509.20328

Spaced Scheduling for Large Language Model Training Amine El hattami, Nicolas Chapados, Christopher Pal Action editor: Colin Raffel https://openreview.net/forum?id=p0KTYl2B9T #scheduling #scheduled #training

really neat clear explainer for the new on “centralizing flows” to theoretically model learning dynamics

Scaling laws don’t just show up in test error — they leave fingerprints in the weight spectrum. In the feature learning regime, we map this connection: phase diagrams of scaling exponents <-> spectral signatures of trained weights. The paper is: arxiv.org/abs/2509.24882

latent space opera

New technical post from Thinky on optimizers but this is the main catch: conditional learning rate per layers. thinkingmachines.ai/blog/modular...

Isaac-01 multimodal model from Perceptron AI - pdf whitepaper github.com/perceptron-a...

New (March) Schmidhuber I missed where they use a carefully engineered layer to track the information gained by each (prediction) token for solving problems that require computation. Hidden state is predictive of (a? not necessarily minimal?) description length.

the Perceptron folks are sharing design specs of their approach to serialize multimodal data as interleaved events www.perceptron.inc/blog/tensors...

Google's Mixboard 💡🧑‍🎨 an experimental, AI-powered concepting board. Designed to help you explore, visualize, and refine your ideas and powered by our latest image generation model Only available in U.S. blog.google/technology/g...

We've hired some *fantastic* researchers but our startup is still looking for 2-3 more people with skills in ML/RL/LLMs. If you'd like to work on some transformative applied problems, hit me up. We'll be launching publicly soon too...

Surprise! A second leaflet on private data in AT, this time exploring some schemes that might be used to implement shared-private data.

Qwen drops Image Edit: apache 2 that feels on par with Nano Banana qwen.ai/blog?id=7a90... Qwen3-Omni: unified image, text, audio and video, like GPT-4o huggingface.co/Qwen/Qwen3-O... Qwen3-TTS: multi-timbre, multi-lingual TTS qwen.ai/blog?id=b426...

I made an agent that unsubscribes you from spam. It controls a web browser to navigate an unsubscribe page. The agent runs JavaScript to perform actions on the page, and gets to see a screenshot after each action it takes. Blog post: blog.aqnichol.com/2025/09/20/u...

Policy churn: maybe epsilon greedy doesn’t matter that much because the Q value argmax action changes constantly arxiv.org/abs/2206.00730

Today my azure A100 quota came through as a little treat. It took about two weeks to get the quota approved…which isn’t bad, I’m not complaining. It’s so great to be installing nvidia drivers again

I've been nerdsniped by the idea of Semantic IDs. Here's the result of my training runs: • RQ-VAE to compress item embeddings into tokens • SASRec to predict the next item (i.e., 4-tokens) exactly • Qwen3-8B that can return recs and natural language! eugeneyan.com/writing/sema...

On the tradeoffs of state space models and transformers (Albert Gu July 2025, Goomba Lab blog) goombalab.github.io/blog/2025/tr...

🎥 Videos of our invited talks and the panel discussion are now also available on YouTube: www.youtube.com/@tokenizatio... ▶️

Great overview of concepts in parallelism for training large models blog.ezyang.com/2025/08/the-...

🎥 Videos from our Tokenization Workshop are now live! Watch invited talks, panel discussions, and the best paper presentation at icml.cc/virtual/2025... #Tokenization #NLP #LLMs

A Survey on Diffusion Language Models Paper: arxiv.org/abs/2508.10875 Repo: github.com/VILA-Lab/Awe...

I actually cracked it long ago but I refuse to sell out. I’ll drop a hint though: numbers

In Chicago next week and interested in incentives around data sharing? Come by our workshop at TTIC! sites.google.com/ttic.edu/inc... We have an incredible lineup of speakers, panelists, and posters, and financial aid apps are still open.

Newsletter: In a media landscape dominated by algorithmic feeds that aim to manipulate and extract, sometimes the most radical thing you can do is choose to read what you want, when you want, without anyone watching over your shoulder. Here’s how to use RSS. www.citationneeded.news/curate-with-...

last day at quillbot today! it’s been a great run

Still not a lot of ML talk on bsky (at least in my feed), hence paper Sunday: my two most interesting recent reads - H Nets arxiv.org/abs/2507.07955 - Energy Based Transformers arxiv.org/abs/2507.02092

Amazingly informative.

multiscale & no frozen tokenizer! Dynamic chunking for end-to-end hierarchical sequence modeling (Hwang, Wang, and Gu) arxiv.org/abs/2507.07955

gm bluesky. It's blue Wednesday so I'm posting over here. I guess. You probably heard about Mecca Hitler and think that that would be enough to drive people off of X but it won't. Not without group chats on bluesky, because the internet actually is a dark forest now

I used Veo 3 to solve Shakespearean tragedies as pictured in classical art. (sound on)

Ah fuck we got the Philip K Dick future 😞

getting in an arms race with LLMs over em-dashes by using dashes so long ——— so very long ——— that they have never appeared in training data

Created an autogenerated list of the accounts I've liked most this past month! (Top-level posts only) Hoping this can serve as a sort of rolling starter pack for accounts I enjoy.

📦 google-gemini / cookbook ⭐ 13,617 (+42) 🗒 Jupyter Notebook Examples and guides for using the Gemini API