In our new preprint, we explain how some salient features of representational geometry in language modeling originate from a single principle - translation symmetry in the statistics of data. arxiv.org/abs/2602.150... With Dhruva Karkada, Daniel Korchinski, Andres Nava, & Matthieu Wyart.

Dhruva Karkada, Daniel J. Korchinski, Andres Nava, Matthieu Wyart, Yasaman Bahri: Symmetry in language statistics shapes the geometry of model representations https://arxiv.org/abs/2602.15029 https://arxiv.org/pdf/2602.15029 https://arxiv.org/html/2602.15029

How do diverse context structures reshape representations in LLMs? In our new work, we explore this via representational straightening. We found LLMs are like a Swiss Army knife: they select different computational mechanisms reflected in different representational structures. 1/

Why isn’t modern AI built around principles from cognitive science or neuroscience? Starting a substack (infinitefaculty.substack.com/p/why-isnt-m...) by writing down my thoughts on that question: as part of a first series of posts giving my current thoughts on the relation between these fields. 1/3

I'll be missing NeurIPS this year, but we have two conference papers on the dynamics of learning and the structure of data in language modeling, a new direction I'm excited about: arxiv.org/abs/2502.09863 and arxiv.org/abs/2505.18651.

Very excited to lead this new @simonsfoundation.org collaboration on the physics of learning and neural computation to develop powerful tools from physics, math, CS, stats, neuro and more to elucidate the scientific principles underlying AI. See our website for more: www.physicsoflearning.org

I'm looking forward to giving a talk tomorrow morning at the ICML workshop on High-Dimensional Learning Dynamics (HiDL) sites.google.com/view/hidimle.... Come by at 9 am!

Excited to be at the APS March Meeting this year! @apsphysics.bsky.social I'll be giving a talk in the Tues afternoon session MAR-J58, Physics of Learning & Adaptation I.

A wide range of insightful and inspiring talks today at #ML4PS @neuripsconf.bsky.social , including @yasamanbb.bsky.social , Thea Aarrestad, and @annalenakofler.bsky.social