I think Fodor & Pylyshyn's 1988 paper is possibly the most mischaracterized paper in the history of cognitive science. It's often cited as arguing that neural networks cannot achieve systematicity, compositionality, and productivity. But that's not what they actually argue...

🚨New preprint! In-context learning underlies LLMs’ real-world utility, but what are its limits? Can LLMs learn completely novel representations in-context and flexibly deploy them to solve tasks? In other words, can LLMs construct an in-context world model? Let’s see! 👀

I'm excited to share that this paper was accepted at ICLR 2026! We show that language models encode one of the most basic ingredients of a world model: the ability to distinguish plausible from implausible states. Check out the paper for more details! See you in Rio! Paper: arxiv.org/abs/2507.12553

I had a great time helping out on this project with @jennhu.bsky.social and Michael Franke! If you're interested in the intersection of interpretability and cogsci, check it out!