In today's Generative AI lecture, we primarily discuss scaling laws and the key factors that go into building large-scale foundation models. Slides: www.cs.cmu.edu/~mgormley/co... Full course info: bsky.app/profile/anay...

Congratulations to my Ph.D. student Reece Keller for winning the best talk award at #CRSy25 on our project building the first task-optimized autonomous agent that predicts whole-brain data! Check out the post below for other cool talks!! Detailed summary: bsky.app/profile/reec...

Congrats to this year's Nobel Prize winners! Philippe's seminal work is in fact what our recent closed form UBI AI capability threshold builds on: bsky.app/profile/anay...

My ILIAD ’25 talk, “Intrinsic Barriers & Pathways to Alignment”: why “aligning to all human values” provably can’t work, why reward hacking is inevitable in large state spaces, & how small value sets bypass “no free lunch” limits to yield formal corrigibility. www.youtube.com/watch?v=Oajq...

A nice application of our NeuroAI Turing Test! Check out @ithobani.bsky.social's thread for more details on comparing brains to machines!

Honored to be quoted in this @newsweek.com article discussing how AI could accelerate the need for UBI. Read more here: www.newsweek.com/ai-taking-jo...

In today's Generative AI lecture, we talk about all the different ways to take a giant auto-complete engine like an LLM and turn it into a useful chat assistant.

In today's Generative AI lecture, we discuss the 4 primary approaches to Parameter-Efficient Fine-Tuning (PEFT): subset, adapters, Prefix/Prompt Tuning, and Low-Rank Adaptation (LoRA). We show each of these amounts to finetuning a different aspect of the Transformer.

1/6 Recent discussions (e.g. Rich Sutton on @dwarkesh.bsky.social’s podcast) have highlighted why animals are a better target for intelligence — and why scaling alone isn’t enough. In my recent @cmurobotics.bsky.social seminar talk, “Using Embodied Agents to Reverse-Engineer Natural Intelligence”,

Excited to have this work accepted as an *oral* to NeurIPS 2025!

Excited to have this work accepted to NeurIPS 2025! See you all in San Diego!

In today's Generative AI lecture, we discuss how to implement Diffusion Models and go through their derivation. Next time, we discuss their deeper relationships with variational inference :) Slides: www.cs.cmu.edu/~mgormley/co... Full course info: bsky.app/profile/anay...

In today's Generative AI lecture, we discuss Generative Adversarial Networks (GANs) & review probabilistic graphical models (PGMs) as a prelude to Diffusion models and VAEs, which we will discuss next time! Slides: www.cs.cmu.edu/~mgormley/co... Full course info: bsky.app/profile/anay...

In today's Generative AI lecture, we cover Vision Transformers (as well as the broader notion of Encoder-Only Transformers). We also explain the historical throughline to some of these ideas, inspired by Nobel-prize-winning observations in neuroscience!

In today's Generative AI lecture, we give an overview of the pre-training/post-training pipeline, and discuss modern Transformer implementations, from Rotary Position Embeddings (RoPE) to Grouped Query Attention (GQA) to Sliding Window Attention.

Cool UBI simulator made by the AI+Wellbeing Institute based on our paper! www.ai-well-being.com/building-our...

In today's Generative AI lecture, we cover how to train a Transformer Language Model, as well as what makes it efficient at learning in order to scale to GPT levels—covering key-value caching & tokenizers, among other things:

If you’re attending ILIAD, I’ll be presenting this work online tomorrow from 11 am - 12 pm PT! www.iliadconference.com

This semester, Matt Gormley & I are co-teaching CMU's Generative AI course! Today we discussed the Transformer architecture & Multi-Headed Attention. Follow along 👇 if you want to learn more about the tech that's powering today's AI, from ChatGPT to reasoning models to agents!

Lab website finally up! anayebi.github.io/group/

1/ Updated now with nearly tight lower bounds—i.e., proofs showing when alignment becomes intractable, even for ideal agents. Key AI safety takeaways: 🧠 Too many values ⇒ makes alignment intractable 👁 Task-space growth ⇒ oversight failure (continued below 🧵👇)

1/ How do we build AI systems that are corrigible—shut down when asked, tell the truth, preserve oversight—and still do something useful? We give the first provable framework that makes it implementable—unlike RLHF or Constitutional AI, which can fail when goals conflict. 🧵👇

🚀 New Open-Source Release! PyTorchTNN 🚀 A PyTorch library for biologically-inspired temporal neural nets: unrolling computation through time. Integrates with our recent Encoder-Attender-Decoder, which flexibly combines models (Transformer, SSM, RNN) since no single one fits all sequence tasks. 🧵👇

As someone who loves our National Parks, this is genuinely concerning. Please RT to spread the word!

1/ Very much resonate w/ Sergey Levine here: we need agents that autonomously learn *in* the world. I've long argued online adaptive autonomy & memory are critical—1 of 4 essential challenges to build agents truly matching the brain’s capabilities. 👇 Thread on some of our recent efforts:

If you're at the #FENS regional meeting this week in Oslo, I'll be giving a talk on Wednesday on our NeurIPS 2021 Spotlight work recovering MEC's heterogeneity of cell types via task-optimization, along with bridging it with some new perspectives to improve AI agents w/ memory.

Super cool—a Japanese YouTube channel made a clear 10-min English summary of my AI UBI paper! Check it out here: www.youtube.com/watch?v=HGQi...

hot damn this is so fucking cool. now this, THIS is how you get something meaningfully described as "AI"

absolutely wild stuff, read the whole thread. where my AI ethicists at? we got a live one here.

Thanks Lana! I really can't emphasize how surreal it was for all of us to see it swim for the first time... Shout out to my MS Robotics student, Alyn Kirsch Tornell, for designing such a realistic skin!

Our first NeuroAgent! 🐟🧠 Excited to share new work led by the talented @reecedkeller.bsky.social, showing how autonomous behavior and whole-brain dynamics emerge naturally from intrinsic curiosity grounded in world models and memory. Check it out here! 👇

Check out our new work on making robots process touch more like brains! Surprisingly, ConvRNNs best matched mouse cortex—passing the NeuroAI Turing Test. We also developed tactile-specific SSL augmentations and an Encoder-Attender-Decoder framework unifying ConvRNNs, SSMs & Transformers.

Can a Universal Basic Income (UBI) become feasible—even if AI fully automates existing jobs and creates no new ones? We derive a closed-form UBI threshold tied to AI capabilities that suggests it's potentially achievable by mid-century even under moderate AI growth assumptions:

Last lecture of the semester! This time on biological learning rules & what's next: agents. We also cover how agents are needed to identify learning rules in the brain, and the core AI challenges needed to build such agents.

As a long-time fan of Paul Middlebrooks' "Brain Inspired" podcast, it was an honor to be invited on to talk about NeuroAgents, our update to the Turing Test, and AI safety at the end. Coincidentally recorded on my birthday, no less! Check it out here 👇

If you want to learn more about the brain science 🧠 origins of Transformers & RNNs, check out today's lecture on memory! Lots more puzzles remain to get scalable memory architectures into AI agents, which is the topic of the next class's debate:

Aran Nayebi (@anayebi.bsky.social): Why NeuroAI Needs NeuroAgents www.youtube.com/watch?v=s3kl...

📢 Join us Thursday, April 3rd | 5–6 PM EST for the first NeuroAI Emerging Leaders Webinar with @anayebi.bsky.social 🧠 Title: Using Embodied Agents to Help Answer “Why” Questions in Systems Neuroscience For further updates, visit: 🔗 neuroai-workshop.github.io//NeuroAI_Web... @neuro-ai.bsky.social

A very well-written article from 2020 (rare these days!) on the imminent societal impact even weakly capable AI will have, by the late Stanford philosopher Ken Taylor. Full article here: www.bostonreview.net/articles/ken...

A prescient discussion on when neural computations are useful & brain-ANN comparisons from Teuvo Kohonen's classic "Self-Organizing Maps" book. A daily reminder that people have been thinking about the relationship between the brain & AI for a long time :)

I'll be speaking as part of the Neuroscience Unleashed summer school in July. Lots of great speakers lined up (see below). Feel free to register & spread the word if interested!

In this lecture, we cover “World Models” & how they might be constructed in brains. It’s an area with many open questions and perhaps the biggest open problem in AI at the moment, likely requiring new models rather than purely more scale. Here we list some possible future directions to match humans:

If you're interested in AI alignment, I'll be giving a talk on this recent work at *5:30 pm ET today*. Livestreamed here: app.ploutos.dev/streams/mach...

Had a lot of fun on this podcast about the practical challenges of scaling (especially in Embodied AI), NeuroAI, what to expect in the future, and advice for students getting into the field. Check it out here! www.youtube.com/watch?v=ZRo-...

What should count as a good model of intelligence? AI is advancing rapidly, but how do we know if it captures intelligence in a scientifically meaningful way? We propose the *NeuroAI Turing Test*—a benchmark that evaluates models based on both behavior and internal representations.

In this lecture, we discuss task-optimized models of the motor system. We end on what I call the *Motor Task Complexity Conjecture*: namely, is the motor system evolved to be "simplicity constrained", or are our tasks not yet fully constraining the dynamics we observe?

Are there fundamental barriers to AI alignment once we develop generally-capable AI agents? We mathematically prove the answer is *yes*, and outline key properties for a "safe yet capable" agent. 🧵👇 Paper: arxiv.org/abs/2502.05934

Honored to be part of a feature in The Transmitter! Come learn more about what our lab does in this article!

One thing I'm finding is that students have a native fluidity of mapping ANNs onto brains. E.g. when discussing distributed vs. localized representations in class today, many would draw analogies to current language models & how they process information.

If an LLM is "humanity's mirror", then post-training is how we filter out the reflections we'd rather not see 🤳