Another intern opening on our team, for a project I’ll be involved in (deadline soon!)

Last month I co-taught a class on diffusion models at MIT during the IAP term: www.practical-diffusion.org In the lectures, we first introduced diffusion models from a practitioner's perspective, showing how to build a simple but powerful implementation from the ground up (L1). (1/4)

Paper🧵 (cross-posted at X): When does composition of diffusion models "work"? Intuitively, the reason dog+hat works and dog+horse doesn’t has something to do with independence between the concepts being composed. The tricky part is to formalize exactly what this means. 1/

finally managed to sneak my dog into a paper: arxiv.org/abs/2502.04549

nice idea actually lol: “Periodic cooking of eggs” : www.nature.com/articles/s44...

Reminder of a great dictum in research, one of 3 drilled into us by my PhD supervisor: "Don't believe anything obtained only one way", for which the actionable dictum is "immediately do a 2nd independent test of something that looks interesting before in any way betting on it". Its a great activity!

I’ve been in major denial about how powerful LLMs are, mainly bc I know of no good reason for it to be true. I imagine this was how deep learning felt to theorists the first time around 😬

Last year, we funded 250 authors and other contributors to attend #ICLR2024 in Vienna as part of this program. If you or your organization want to directly support contributors this year, please get in touch! Hope to see you in Singapore at #ICLR2025!

The thing about "AI progress is hitting a wall" is that AI progress (like most scientific research) is a maze, and the way you solve a maze is by constantly hitting walls and changing directions.

Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥 📄 arxiv.org/abs/2410.23054 🛠️ github.com/apple/ml-act 0/9 🧵

📢 My team at Meta (including Yaron Lipman and Ricky Chen) is hiring a postdoctoral researcher to help us build the next generation of flow, transport, and diffusion models! Please apply here and message me: www.metacareers.com/jobs/1459691...

Giving a short talk at JMM soon, which might finally be the push I needed to learn Lean…

Just read this, neat paper! I really enjoyed Figure 3 illustrating the basic idea: Suppose you train a diffusion model where the denoiser is restricted to be "local" (each pixel i only depends on its 3x3 neighborhood N(i)). The optimal local denoiser for pixel i is E[ x_0[i] | x_t[ N(i) ] ]...cont

LLMs dont have motives, goals or intents, and so they wont lie or deceive in order to obtain them. but they are fantastic at replicating human culture, and there, goals, intents and deceit abound. so yes, we should also care about such "behaviors" (outputs) in deployed systems.

One #postdoc position is still available at the National University of Singapore (NUS) to work on sampling, high-dimensional data-assimilation, and diffusion/flow models. Applications are open until the end of January. Details: alexxthiery.github.io/jobs/2024_di...

“Should you still get a PhD given o3” feels like a weird category error. Yes, obviously you should still have fun and learn things in a world with capable AI. What else are you going to do, sit around on your hands?

Catch our talk about CFG at the M3L workshop Saturday morning @ Neurips! I’ll also be at the morning poster session, happy to chat

Found slides by Ankur Moitra (presented at a TCS For All event) on "How to do theoretical research." Full of great advice! My favourite: "Find the easiest problem you can't solve. The more embarrassing, the better!" Slides: drive.google.com/file/d/15VaT... TCS For all: sigact.org/tcsforall/

When a bunch of diffusers sit down and talk shop, their flow cannot be matched😎 It's time for the #NeurIPS2024 diffusion circle! 🕒Join us at 3PM on Friday December 13. We'll meet near this thing, and venture out from there and find a good spot to sit. Tell your friends!

I used to not really care to take pictures of myself, because what would I do with them, but over the break I had a fun training a bunch of Flux LoRAs on myself (surprisingly easy). And now I have renewed interest in profile photos.

Help, I'm going to NeurIPS next week! If you are presenting something that you think I should check out, pls drop the deets in thread (1-2 sentence description + where to find you)! 🙏

An aspect of flow matching which I find a bit interesting is that it is covariant under affine changes of coordinate (c.f. optimal transport, which need not be). This allows for a few nice WLOGs, which I imagine have more applications than I realise.

Blog post link: diffusionflow.github.io/ Despite seeming similar, there is some confusion in the community about the exact connection between the two frameworks. We aim to clear up the confusion by showing how to convert one framework to another, for both training and sampling.

Sander poses a good question in this thread: Why is there so little variance in learning of diffusion models? Ie, why do different models trained on [random samples from] the same data distribution end up learning *nearly-identical* Noise --> Data function? Some background & speculations:

An intro to TOC class should indeed focus on "useful" ideas. But "useful" is often incredibly narrowlt defined. TOC is more way than its theorems, it's a way of modeling & thinking about problems that is applicable much more broadly across sciences & engg. Wigderson argues better than I could.

IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.

Our group at Google DeepMind is now accepting intern applications for summer 2025. Attached is the official "call for interns" email; the links and email aliases that got lost in the screenshot are below.

Vague ideas that may not be strictly true are called “intuitions” or “heuristics” or sometimes even “principles” or “laws” (depending on the field). And in the long term such ideas can be extremely impactful.

A list of all the stats/modeling/ML/data starter packs I've seen (26+ and counting):

(it’s that time of year again..) Wonder if it’d help to make AC names visible to reviewers, and encourage both to keep in touch. That way, outstanding reviewers have an “in” to collab with more senior members of the community — similar to how ugrads get into research by being exceptional in class

despite my best efforts over the years, the biggest factor in my mood is often still “how well is the research going”

As my first post on this platform, allow me to advertise the RL theory lecture notes I have been developing with Sasha Rakhlin: arxiv.org/abs/2312.16730 (shameless repost of my pinned tweet)

I wrote a Part IV postscript to my job market blog post to add what I've learned as faculty. TL;DR: No one is out to get you. For anything not going your way, it's probably due to people being busy or bureaucracy. And there are probably people working very hard for you behind the scenes regardless.