🚨 New preprint! TL;DR: Backtracking is not the "holy grail" for smarter LLMs. It’s praised for helping models “fix mistakes” and improve reasoning—but is it really the best use of test-time compute? 🤔

🎉Microsoft Research New England is hiring a predoctoral research assistant to work with @nancybaym.bsky.social, Tarleton Gillespie, and @marylgray.bsky.social on issues related to the dynamics of technology and society. 🎉 socialmediacollective.org/2025/01/22/s...

Transformer LMs get pretty far by acting like ngram models, so why do they learn syntax? A new paper by sunnytqin.bsky.social, me, and @dmelis.bsky.social illuminates grammar learning in a whirlwind tour of generalization, grokking, training dynamics, memorization, and random variation. #mlsky #nlp