Cengiz Pehlevan (@cpehlevan.bsky.social)

Solvable Model of In-Context Learning Using Linear Attention - Kempner Institute Attention-based architectures are a powerful force in modern AI. In particular, the emergence of in-context learning enables these models to perform tasks far beyond the original next-token prediction... https://kempnerinstitute.harvard.edu/research/deeper-learning/solvable-model-of-in-context-learning-using-linear-attention/