Why does noisy gradient-descent train neural nets? This fundamental question in ML remains unclear. In our hugely revised draft my student @dkumar9.bsky.social gives the full proof that a form of noisy-GD, Langevin Monte-Carlo (#LMC), can learn arbitrary depth 2 nets. arxiv.org/abs/2503.10428

Noisy gradient descent has attracted a lot of attention in the last few years as a mathematically tractable model of actual deep-learning algorithms. In my recent work with @anirbit.bsky.social and Samyak Jha (arxiv.org/abs/2503.10428), we prove noisy gradient descent learns neural nets.