A great @quantamagazine.bsky.social article on our theory of creativity in convolutional diffusion models lead by @masonkamb.bsky.social See also our paper with new results in version 2: arxiv.org/abs/2412.20292 to be presented as an oral at @icmlconf.bsky.social #icml25

Honored to have had my recent work with @suryaganguli.bsky.social on the mechanisms behind creativity in diffusion models featured in this lovely article by Webb Wright for Quanta magazine!

NEW: Mason Kamb ‪( @masonkamb.bsky.social ‬) from @stanford.edu‬ presents a predictive theory of combinatorial creativity in diffusion models. Watch the video: youtu.be/DP_kGt0-2cg #NeuroAI2025 #AI #ML #NeuroAI

The DOGE etc. damage to US science will have enormous effects that will linger for decades. But they will be sufficiently gradual and diffuse that people who want to pretend the cause wasn't obvious will be able to do so.

In another blow to legacy media, I'm hearing that the Trump administration plans to remove The Atlantic from its war-plans group chat. The outlet will be replaced in the chat by the Gateway Pundit.

real instructive that just by paying attention to the background hum of regular small plane crashes the media has created a perception of a sharp increase

Finally got to reading the fascinating & excellent paper by Kamb and Ganguli, which makes a significant contribution to diffusion/GenAI literature & will likely become one of the most-cited works in this space. Unlike many "theoretical" ML studies, theirs is high-dimensional and practical.. 1/n

Our new paper! "Analytic theory of creativity in convolutional diffusion models" lead expertly by @masonkamb.bsky.social arxiv.org/abs/2412.20292 Our closed-form theory needs no training, is mechanistically interpretable & accurately predicts diffusion model outputs with high median r^2~0.9

Excited to finally share this work w/ @suryaganguli.bsky.social Tl;dr: we find the first closed-form analytical theory that replicates the outputs of the very simplest diffusion models, with median pixel wise r^2 values of 90%+. arxiv.org/abs/2412.20292