Great blog post on rotary position embeddings (RoPE) in more than one dimension, with interactive visualisations, a bunch of experimental results, and code!

I blog and give talks to help build people's intuition for diffusion models. YouTubers like @3blue1brown.com and Welch Labs have been a huge inspiration: their ability to make complex ideas in maths and physics approachable is unmatched. Really great to see them tackle this topic!

Hello #ICML2025👋, anyone up for a diffusion circle? We'll just sit down somewhere and talk shop. 🕒Join us at 3PM on Thursday July 17. We'll meet here (see photo, near the west building's west entrance), and venture out from there to find a good spot to sit. Tell your friends!

Diffusion models have analytical solutions, but they involve sums over the entire training set, and they don't generalise at all. They are mainly useful to help us understand how practical diffusion models generalise. Nice blog + code by Raymond Fan: rfangit.github.io/blog/2025/op...

Here's the third and final part of Slater Stich's "History of diffusion" interview series! The other two interviewees' research played a pivotal role in the rise of diffusion models, whereas I just like to yap about them 😬 this was a wonderful opportunity to do exactly that!

The ML for audio 🗣️🎵🔊 workshop is back at ICML 2025 in Vancouver! It will take place on Saturday, July 19. Featuring invited talks from Dan Ellis, Albert Gu, James Betker, Laura Laurenti and Pratyusha Sharma. Submission deadline: May 23 (Friday next week) mlforaudioworkshop.github.io

I am very happy to share our latest work on the information theory of generative diffusion: "Entropic Time Schedulers for Generative Diffusion Models" We find that the conditional entropy offers a natural data-dependent notion of time during generation Link: arxiv.org/abs/2504.13612

One weird trick for better diffusion models: concatenate some DINOv2 features to your latent channels! Combining latents with PCA components extracted from DINOv2 features yields faster training and better samples. Also enables a new guidance strategy. Simple and effective!

New blog post: let's talk about latents! sander.ai/2025/04/15/l...

Amazing interview with Yang Song, one of the key researchers we have to thank for diffusion models. The most important lesson: be fearless! The community's view on score matching was quite pessimistic at the time, he went against the grain and made it work at scale! www.youtube.com/watch?v=ud6z...

🥁Introducing Gemini 2.5, our most intelligent model with impressive capabilities in advanced reasoning and coding. Now integrating thinking capabilities, 2.5 Pro Experimental is our most performant Gemini model yet. It’s #1 on the LM Arena leaderboard. 🥇

We are hiring on the Generative Media team in London: boards.greenhouse.io/deepmind/job... We work on Imagen, Veo, Lyria and all that good stuff. Come work with us! If you're interested, apply before Feb 28.

Great interview with @jascha.sohldickstein.com about diffusion models! This is the first in a series: similar interviews with Yang Song and yours truly will follow soon. (One of these is not like the others -- both of them basically invented the field, and I occasionally write a blog post 🥲)

📢PSA: #NeurIPS2024 recordings are now publicly available! The workshops always have tons of interesting things on at once, so the FOMO is real😵‍💫 Luckily it's all recorded, so I've been catching up on what I missed. Thread below with some personal highlights🧵

Why do diffusion models generalise at all? It's not obvious that they would. It turns out underfitting plays an important role, as well as the architectural inductive biases of locality and translation equivariance. What other kinds of symmetry and structure could we hardcode? 🤔

I ran across a busy Sander at a #neurips party with a similar question - he was still patient enough to explain stuff. This talk further clarifies a good amount of my doubts. Recommend watching if you're working on diffusion / LLMs for generation!

The recording of my #NeurIPS2024 workshop talk on multimodal iterative refinement is now available to everyone who registered: neurips.cc/virtual/2024... My talk starts at 1:10:45 into the recording. I believe this will be made publicly available eventually, but I'm not sure when exactly!

Here's Veo 2, the latest version of our video generation model, as well as a substantial upgrade for Imagen 3 🧑‍🍳🚢 (Did I mention we are hiring on the Generative Media team, btw 👀) blog.google/technology/g...

I've been getting a lot of questions about autoregression vs diffusion at #NeurIPS2024 this week! I'm speaking at the adaptive foundation models workshop at 9AM tomorrow (West Hall A), about what happens when we combine modalities and modelling paradigms. adaptive-foundation-models.org

If you're at #NeurIPS2024, join us this afternoon to talk diffusion, flows and all that jazz. Be there or be non-circular!

When a bunch of diffusers sit down and talk shop, their flow cannot be matched😎 It's time for the #NeurIPS2024 diffusion circle! 🕒Join us at 3PM on Friday December 13. We'll meet near this thing, and venture out from there and find a good spot to sit. Tell your friends!

“On a log-log plot, my grandmother fits on a straight line.” -Physicist Fritz Houtermans There's a lot of truth to this. log-log plots are often abused and can be very misleading 1/5

Better VQ-VAEs with this one weird rotation trick! I missed this when it came out, but I love papers like this: a simple change to an already powerful technique, that significantly improves results without introducing complexity or hyperparameters.

There is a lot of great writing on flow matching out there all of a sudden! This post clarifies the connection with diffusion models -- they are essentially two different ways to describe the same class of models.

In arxiv.org/abs/2303.00848, @dpkingma.bsky.social and @ruiqigao.bsky.social had suggested that noise augmentation could be used to make other likelihood-based models optimise perceptually weighted losses, like diffusion models do. So cool to see this working well in practice!

One of these is not like the others 😁

Sander poses a good question in this thread: Why is there so little variance in learning of diffusion models? Ie, why do different models trained on [random samples from] the same data distribution end up learning *nearly-identical* Noise --> Data function? Some background & speculations:

The link between diffusion models and optimal transport is still a bit of an enigma to me. One thing that's clear: different diffusion models trained on similar datasets tend to recover similar mappings. If these are generally not OT, in what sense are they optimal instead?

Our group at Google DeepMind is now accepting intern applications for summer 2025. Attached is the official "call for interns" email; the links and email aliases that got lost in the screenshot are below.

We just dropped CAT4D, text to dynamic 3D models that you can render in real time. Not posting a video because Bluesky is garbage in this respect; go straight to the real time viewer on a desktop browser and look around. The cat kneading dough is my favorite. cat-4d.github.io

Asking the following earnestly: what is the strongest case for GANs standing the "test of time"? Are they important 10 years later in modern ML research? How have they influenced the way we think about generative models today?

IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.

GANs fundamentally changed generative vision. And even if people now use diffusion/flow models for many generative tasks, they typically do so in a latent space, where the decoder often effectively is a conditional GAN still.

The first generative AI methods with impressive scaling performance were GANs. In other words, GANs started the generative AI field, and were the dominant method until replaced by diffusion models (whose research benefited from knowing that one could scale).

Amazing blog post on flow matching, stunning visuals! It also makes the connection with normalising flows crystal clear. Incredible effort!

Arxiv sharing reminder pdf ❌ abs ✅

I'll be at NeurIPS in Vancouver✈️, where I'm speaking at the Adaptive Foundation Models workshop on Saturday, and I'm sure a diffusion circle will coalesce at some point during the week! Also heading to SF the week before (next week!), let me know if anything cool is happening👀

Posting a call for help: does anyone know of a good way to simultaneously treat both POTS and Ménière’s disease? Please contact me if you’re either a clinician with experience doing this or a patient who has found a good solution. Context in thread

We know that diffusion models learn interesting representations during training, but can we increase their generation performances by distilling representations learned by self-supervised encoders? 👀 The REPA article by Sihyun Yu et al. shows better FID + faster convergence arxiv.org/abs/2410.06940

I was contemplating giving up on Threads and sticking with 🐦 and 🦋, where most of the action seems to be. Looks like they just made the decision for me 😂

Here's another oldie from 2020: an entire blog post about the weirdness of high-dimensional probability distributions. With generative modelling having taken off as it has, understanding typicality and its implications is all the more important, but it still isn't talked about much in ML circles!

When I will respond to your email

Discrete diffusion has become a very hot topic again this year. Dozens of interesting ICLR submissions and some exciting attempts at scaling. Here's a bibliography on the topic from the Kuleshov group (my open office neighbors). github.com/kuleshov-gro...

Started a list of some researchers working on image/video generation. (Not comprehensive at all) Reply with a paper link and TLDR to get added to the list! I request all grad students to not feel imposter-y and just reply if you work in this field! #computervision #diffusion go.bsky.app/SP1uWoE

Story time! Some of the most productive time I spent during my PhD was, ironically, when Kaggle competitions distracted me from my research😁 I learnt to assess and reimplement ideas from papers, and how to properly train neural nets. (1/6)

About a decade (😱) ago, I spent the summer at Spotify working on audio-based music recommendation. Music recommendations are mainly driven by listening patterns, but analysing audio helps address the cold start problem: recommending music before anyone has listened to it! sander.ai/2014/08/05/s...

Take a few minutes to add a profile picture and set your name before using starter packs. You’ll find a lot of people will follow you back!

I created 3 introductory notebooks on Flow Matching models to help get started with this exciting topic! ✨ 1. Annotated Flow Matching paper: github.com/gle-bellier/... 2. Discrete Flow Matching: github.com/gle-bellier/... 3. Minimal FM in Jax: github.com/gle-bellier/...

Fellow #Diffusion and #FlowMatching enthusiasts, any tips for presenting this class of generative models to a broader ML audience? Any examples of slides that you have seen (or made) that you really like? 📊

I've started putting together a starter pack with people working on Speech Technology and Speech Science: go.bsky.app/BQ7mbkA (Self-)nominations welcome!