new blog post! why do LLMs freak out over the seahorse emoji? i put llama-3.3-70b through its paces with the logit lens to find out, and explain what the logit lens (everyone's favorite underrated interpretability tool) is in the process. link in reply!

How Diffusion Models Memorize Juyeop Kim, Songkuk Kim, Jong-Seok Lee tl;dr: classifier-free-guidance is to blame arxiv.org/abs/2509.25705

Very happy this paper got accepted to NeurIPS 2025 as a Spotlight! 😁 Main takeaway: In mechanistic interpretability, we need assumptions about how DNNs encode concepts in their representations (eg, the linear representation hypothesis). Without them, we can claim any DNN implements any algorithm!

really neat clear explainer for the new on “centralizing flows” to theoretically model learning dynamics

Half-serious question: Could one tackle every "language models produce X because Y is in the training data" hypothesis in one go by training a large retrieval transformer, and doing side-by-side evaluations with/without a filter for Y on the retriever? #MLSky

And new paper out: Pleias 1.0: the First Family of Language Models Trained on Fully Open Data How we train an open everything model on a new pretraining environment with releasable data (Common Corpus) with an open source framework (Nanotron from HuggingFace). www.sciencedirect.com/science/arti...

Stepping outside my lane for a bit, NPM really needs trusted publishing/provenance, but I think a per-package "will use secure-context-only features" flag would go a long way too, there are few good reasons for a package to suddenly _start_ using eval(), Fetch, or executing arbitrary commands

As someone interested in cryptography and privacy-preserving measures, it's cool to see differential privacy being applied to ML practically like this - this seems to basically stop memorization of passages from the training set, even allowing for approximate matches (up to 10% edit distance)

We've just released an amazing Embedding model: EmbeddingGemma, the new best-in-class open embedding model! 🚀 🏆 Top multilingual model on MTEB (<500M) 💾 Runs on <200MB RAM ⚙️ Customizable output for on-device use 🧩 Integrated with your favorite tools developers.googleblog.com/en/introduci...

EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model. Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good. Read more: actu.epfl.ch/news/apertus...

As someone who hasn't used MacOS in a minute (since... Sierra?), which MacOS utilities do people on #MLSky recommend? (Homebrew is a given, but not sure which terminal emulators people prefer now, for instance)

Limits of vector search a new GDM paper shows that embeddings can’t represent combinations of concepts well e.g. Dave likes blue trucks AND Ford trucks even k=2 sub-predicates make SOTA embedding models fall apart www.alphaxiv.org/pdf/2508.21038

NLDL 2026 paper submission deadline extended 🤖 ❄️ Due to a large amount of requests, we wish to inform you that the paper submission deadline is extended until September 14th, 23:59 CEST More info about submissions at nldl.org

Interesting argument that the architectural blocks of language models are almost always surjective, implying that any given output text has at least one corresponding input which can generate it (They do note that *finding* that input isn't necessarily easy, and assume you control input embeddings)

Wonderful project which packages ELIZA and other retro chatbots in an OpenAI-compatible API, because we all need a code assistant that can ask "how does the NullReferenceException make you feel?"

Nvidia have released a new set of hybrid Mamba-Transformer language models, but interestingly they've also published the majority of the data used to pretrain them: research.nvidia.com/labs/adlr/NV...

Fun breakdown of the process of implementing a toy TPU in Verilog, starting "from scratch" www.tinytpu.com

Interesting from DeepMind: Live Music Models arxiv.org/pdf/2508.04651 - low-latency music generation models which run at real time and respond to user input, letting you perform style transfer on your own music on the fly

Shower thought: if you only had pen and paper (no calculator), what kind of neural networks could you feasibly still compute predictions with? I think one could make something like the earliest LeNets work, with ReLU activation and some form of weight quantization to simplify things

Today's small but neat trick: setting up a simple web server in Powershell to serve a Marimo notebook to localhost, giving you a light Python + Numpy + Matplotlib setup on any Windows box without any installation or network requirements