Last year Molmo set SOTA on image benchmarks + pioneered image pointing. Millions of downloads later, Molmo 2 brings Molmo’s grounded multimodal capabilities to video 🎥—and leads many open models on challenging industry video benchmarks. 🧵

Oof

RoMa v2 is now out! (github.com/Parskatt/rom..., arxiv.org/abs/2511.15706) Here are the main improvements we made since RoMa:

Excited to share that we'll be presenting four papers at the main conference at ICCV 2025 this week! Come say hi in Honolulu! 👋 Pingchuan, Ming, Felix, Stefan, Timy, and Björn Ommer will be attending.

🤔 What if you could generate an entire image using just one continuous token? 💡 It works if we leverage a self-supervised representation! Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇

🤔 What happens when you poke a scene — and your model has to predict how the world moves in response? We built the Flow Poke Transformer (FPT) to model multi-modal scene dynamics from sparse interactions. It learns to predict the 𝘥𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘰𝘯 of motion itself 🧵👇

“Everyone knows” what an autoencoder is… but there's an important complementary picture missing from most introductory material. In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵

I'm calling it now, GSPO will be the next big hype in LLM RL algos after GRPO. It makes so much more sense intuitively to work on a sequence rather than on a token level when our rewards are on a sequence level.

🎉 Excited to share that our lab has three papers accepted at CVPR 2025! Come say hi in Nashville! 👋 Johannes, Ming, Kolja, Stefan, and Björn will be attending.

If you are interested, feel free to check the paper (arxiv.org/abs/2506.02221) or come by at CVPR: 📌 Poster Session 6, Sunday 4:00 to 6:00 PM, Poster #208

Here's the third and final part of Slater Stich's "History of diffusion" interview series! The other two interviewees' research played a pivotal role in the rise of diffusion models, whereas I just like to yap about them 😬 this was a wonderful opportunity to do exactly that!

#KostasThoughts: Another major conference review drop is around the corner. In baseball, a .300 average is elite. In research, it’s a familiar reality: submitting to top conferences means rejections happen. Keep swinging!

I am very happy to share our latest work on the information theory of generative diffusion: "Entropic Time Schedulers for Generative Diffusion Models" We find that the conditional entropy offers a natural data-dependent notion of time during generation Link: arxiv.org/abs/2504.13612

New blog post: let's talk about latents! sander.ai/2025/04/15/l...

And the CVPR oral decisions are out! (on Openreview)

Introducing VGGT (CVPR'25), a feedforward Transformer that directly infers all key 3D attributes from one, a few, or hundreds of images, in seconds! Project Page: vgg-t.github.io Code & Weights: github.com/facebookrese...

Introducing DaD (arxiv.org/abs/2503.07347), a pretty cool keypoint detector. As this will get pretty long, this will be two threads. The first will go into the RL part, and the second on the emergence and distillation.

The fate of your #CVPR2025 submission

🤔When combining Vision-language models (VLMs) with Large language models (LLMs), do VLMs benefit from additional genuine semantics or artificial augmentations of the text for downstream tasks? 🤨Interested? Check out our latest work at #AAAI25: 💻Code and 📝Paper at: github.com/CompVis/DisCLIP 🧵👇

What I like to do when considering a new dataset is to train a simple classifier and look at 'the most confident errors'. Recently with NICO: Apart from a class, the images have a context, one of them is 'autumn'. There is also a pumpkin class. Surprise surprise, many autumn images contain pumpkins.

Just had an idea

How to schedule a meeting? When you ask for a meeting with others, you are asking for their time. You are asking for their most valuable, finite resource to benefit yourself (e.g., for advice, networking, questions, and opportunities). Here are some tips that I found useful.

Do you like the power of diffusion features for semantic correspondence but dread running an expensive ~1B model to get them? What if you could have even better features at a fraction of the cost? If this sounds enticing, take a look at this paper! ⬇️

Ever wondered if diffusion features could do better without all the noise? 🤔 Turns out they can! We show how adapting the backbone unlocks clean, powerful features for better results across the board. 🚀🧹 Check it out! ⬇️

Blog post link: diffusionflow.github.io/ Despite seeming similar, there is some confusion in the community about the exact connection between the two frameworks. We aim to clear up the confusion by showing how to convert one framework to another, for both training and sampling.

🤔 Can you turn your vision-language model from a great zero-shot model into a great-at-any-shot generalist? Turns out you can, and here is how: arxiv.org/abs/2411.15099 Really excited to this work on multimodal pretraining for my first bluesky entry! 🧵 A short and hopefully informative thread:

Our group at Google DeepMind is now accepting intern applications for summer 2025. Attached is the official "call for interns" email; the links and email aliases that got lost in the screenshot are below.

The ✨ML Internship Feed✨ is here! @serge.belongie.com and I created this feed to compile internship opportunities in AI, ML, CV, NLP, and related areas. The feed is rule-based. Please help us improve the rules by sharing feedback 🧡 🔗 Link to the feed: bsky.app/profile/did:...

After many years, our lab finally has a social media presence at @compvis.bsky.social ! 🥳 Give it a follow, we have some amazing research on generative computer vision coming soon!

me right now..

The auspicious appearance of @jbhuang0604.bsky.social on Bluesky inspires me to look back at my notes on “how to get cool research ideas,” which I started jotting down way back in 2001, at the start of @belongielab.bsky.social at UC San Diego (1/18)

The world doesn’t live on a pixel grid and neither should vision models! Excited to share Moving off-the-Grid (MooG): a video model w/o grid-based representations. MooG learns detached “off-the-grid tokens” that bind to (and track) scene elements as camera & content move. 🧵

In a gratuitous attempt to acquire more followers myself 😁, I've made a start on a "starter pack". Hopefully as more people from 🐦 make it over to 🦋, we can extend this a bit. Suggestions welcome! I've noticed not all accounts seem to be eligible to be added, anyone know what's up with that? 🤔