New Year’s resolution: more personal coding projects this year. Just finished pretraining and instruction fine tuning my own GPT2 scale LLM from scratch in PyTorch. Was so much fun!

Our new paper "High-capacity associative memory in a quantum-optical spin glass." Achieves 7-fold increased capacity thru a quantum optical analog of short term plasticity from neuroscience. Atoms (neurons) couple to motion and photons (synapses)! arxiv.org/abs/2509.12202

In this Stanford news article, @stanfordhai.bsky.social Senior Fellow @suryaganguli.bsky.social argues that universities are essential for understanding how AI works. Here, he outlines three reasons why: news.stanford.edu/stories/2025...

Led by @stanford.edu's @suryaganguli.bsky.social, our Simons Collaboration on the Physics of Learning & Neural Computation will study AI's fundamental scientific principles: www.simonsfoundation.org/simons-foundation-launches-collaboration-on-the-physics-of-learning-and-neural-computation/ #science

A nice starter pack of researchers in the @simonsfoundation.org Collaboration on the Global Brain go.bsky.app/66dLpPs

Open faculty position! We're seeking applicants for a tenure-track faculty position at the junior level (Assistant or untenured Associate Professor) with research programs that exist at the interface between molecular science and computation. Apply here: stanford.io/45MF3Qa

Very excited to lead this new @simonsfoundation.org collaboration on the physics of learning and neural computation to develop powerful tools from physics, math, CS, stats, neuro and more to elucidate the scientific principles underlying AI. See our website for more: www.physicsoflearning.org

Under Trump, NOAA ran the most recent competition for their highly prestigious postdoc program in climate and global change — but funded no one, wasting huge amounts of applicant and reviewer time. It gets a lot worse. Now current postdocs in the program are going unpaid.

A great @quantamagazine.bsky.social article on our theory of creativity in convolutional diffusion models lead by @masonkamb.bsky.social See also our paper with new results in version 2: arxiv.org/abs/2412.20292 to be presented as an oral at @icmlconf.bsky.social #icml25

Honored to have had my recent work with @suryaganguli.bsky.social on the mechanisms behind creativity in diffusion models featured in this lovely article by Webb Wright for Quanta magazine!

Wow - Bengali political ad in the US! I believe this is a first. খুব ভালো

The best part of this job is seeing students graduate and launch their careers! Congrats to Feng Chen, Atsushi Yamamura, Tamra Nebabu, Linnie Wharton and Daniel Kunin. They are all going on to top positions across artificial intelligence, medicine, and physics. Proud of you!

Had fun presenting at @kempnerinstitute.bsky.social on Theories of rapid learning, creativity, and reasoning: of mice and machines. Video: www.youtube.com/watch?v=jLmK... Papers: Learning: www.nature.com/articles/s41... Creativity: arxiv.org/abs/2412.20292 Math Reasoning: arxiv.org/abs/2502.07154

Next was an fascinating talk by @suryaganguli.bsky.social on theories of learning, with some incredible experiments with mice at the @kempnerinstitute.bsky.social www.youtube.com/watch?v=jLmK... (5/18)

Surya Ganguli of @stanford.edu‬ offers computational insights into the working of grid cells in entorhinal cortex. @suryaganguli.bsky.social‬ #NeuroAI2025 #neuroscience

"Senior officials at the US Department of Veterans Affairs have ordered that VA physicians and scientists not publish in medical journals or speak with the public without first seeking clearance from political appointees of Donald Trump, the Guardian has learned."

OPT is the program that allows foreign students graduating with STEM degrees stay and work for up to three years. Because of the oversubscribed H-1B lottery, OPT is one of the main ways that foreign students manage to transition to jobs in the US. Ending it would destroy a major STEM pipeline.

Many recent posts on free energy. Here is a summary from my class “Statistical mechanics of learning and computation” on the many relations between free energy, KL divergence, large deviation theory, entropy, Boltzmann distribution, cumulants, Legendre duality, saddle points, fluctuation-response…

Researchers who have had grants terminated--tell your story! The #DefendResearch team is collecting testimonials about this impact of #censorship on research that we will use as part of a toolkit for contacting elected officials. #NSF, #NEH, #NIH www.defendresearch.org/testimonials

I signed. Please consider signing.

This might be the most useful thing I have come across in social media - a personalized feed of academic papers filtered by your follower network! Highly recommend. #academicsky

So nice to be surrounded by >12000 physicists at @apsphysics.bsky.social summit working hard to understand the nature of our universe and build better tech like solar cells and novel quantum materials. Important to remind the public that our living standards depend on this research.

1. For the past thirty years I've had the best job in the world. 
I've had the opportunity to follow my curiosity; explore the workings of nature and society; mentor students and junior colleagues in the same process; and teach generations of students about it all.

Love this post @michaelnielsen.bsky.social! I may be biased by I like the 2008 version 4 years before where the song is in my native Bengali by the Nobel Prize winner Rabindranath Tagore: www.youtube.com/watch?v=zlfK... A translation of the song can be found here: www.youtube.com/watch?v=Iiwp...

This is a super clear and helpful article explaining why the 15% indirect cost rate is unfair, hurts science, and does NOT increase $ for research: www.science.org/doi/10.1126/...

This letter appeared in the Bloomsburg PA Press Enterprise this morning. I am assured the writer is who he purports to be, an Air Force veteran.

At the DC #standupforscience today. The question was “Why are you here?” She came up with this on her own! (I helped her with the spelling) So proud of her! #standupforscience2025 @standupforscience.bsky.social

My talk on “Neuroscience and AI: a symbiosis” @SimonsInstitute I start with a whirlwind history of metaphors for intelligence, spanning Hippocratic hydraulics, Hobbesian clocks, Cartesian dualism to Turing, telegraphs, Hopfield, and deep learning. simons.berkeley.edu/talks/surya-...

I just wrote a book about how the type of work @markhisted.org is doing is key to breaking through the bottlenecks holding back new treatments for brain and mental disorders (from Alzheimers to depression). Seeing his work in danger makes me ill, and angry. We must @standupforscience.bsky.social.

Please share widely. Firing of excellent scientists by political appointees with zero oversight is outrageous. This requested pushback from congress is essential to maintain scientific independence - a hallmark of any functioning democracy.

Join us! Science Homecoming helps scientists reconnect with communities by writing about the importance of science funding in their hometown newspapers. We’ve mapped every small newspaper in the U.S. and provide resources to get you started. Help science get back home 🧪🔬🧬 🏠 sciencehomecoming.com

Love this quote from Sir Michael Atiyah: "Algebra is the offer made by the devil to a mathematician. The devil says, 'I will give you this powerful machine...all you need to do is give up your soul: give up geometry...' when you do algebra...you stop thinking about meaning, about geometry

It only takes a minute to write to your reps to support the NIH snd medical research. Please consider it. I just wrote. www.votervoice.net/mobile/AACR/...

My @TEDAI2024 talk is out! www.ted.com/talks/surya_... I discuss our work, spanning AI, physics, math & neuroscience, to develop a deeper scientific understanding of intelligence, both to understand biological intelligence and make better artificial intelligence. I focus on 5 gaps:

Don’t work on: Concentration inequalities (math) Interneuron diversity (neuro) Barrier functions (optimization) Thermodynamic equalities (physics) Community detection (stats) Biased estimators (stats) Marginalization in graphical models (math) Set inclusion (math) Traumatic brain injury (neuro)

@suryaganguli.bsky.social gives great talk at LLM workshop at Berkley -LLM<->brain is still a new topic, less progress so far than LLM<->vision -lets train LLM foundation models of specific brain systems and then reverse engineer them -emerging paradigm: read-write experiments in brains+machines

Am organizing a workshop this week @SimonsInstitute on Large language models, cognitive science, linguistics, neuroscience and theory. Many excellent speakers. Can register here for livestream! simons.berkeley.edu/workshops/ll...

This is poignant. The person in this cartoon could very well be me (hopefully!)

My #NeurIPS204 workshop keynote: "Analytic theory of creativity of convolutional diffusion models" now available: neurips.cc/virtual/2024... It reports on work with @MasonKamb in this paper: arxiv.org/abs/2412.20292

📢PSA: #NeurIPS2024 recordings are now publicly available! The workshops always have tons of interesting things on at once, so the FOMO is real😵‍💫 Luckily it's all recorded, so I've been catching up on what I missed. Thread below with some personal highlights🧵

Just read this, neat paper! I really enjoyed Figure 3 illustrating the basic idea: Suppose you train a diffusion model where the denoiser is restricted to be "local" (each pixel i only depends on its 3x3 neighborhood N(i)). The optimal local denoiser for pixel i is E[ x_0[i] | x_t[ N(i) ] ]...cont

Our new paper! "Analytic theory of creativity in convolutional diffusion models" lead expertly by @masonkamb.bsky.social arxiv.org/abs/2412.20292 Our closed-form theory needs no training, is mechanistically interpretable & accurately predicts diffusion model outputs with high median r^2~0.9

Really fantastic work by @masonkamb.bsky.social on an analytic, mechanistically interpretable, and predictive theory of creativity in convolutional diffusion models! arxiv.org/pdf/2412.20292

Our new paper "Fooling LLM graders into giving better grades through neural activity guided adversarial prompting" lead expertly by Atsushi Yamamura shows how even in closed models like Gemini, we can add a small adversarial suffix to an essay to get unreasonably high scores arxiv.org/abs/2412.15275

Wonderful stories about the late Lubert Stryer’s life and science. He was such amazing human being. When I just started as an assistant professor he took great interest in my work (he did this with everyone not just me!) and he asked incredible questions. The world lost a giant.

We are pleased to announce our first ever Kempner Institute Symposium, Frontiers in NeuroAI, a two-day event in Allston, MA. Virtual and in-person options available. Learn more: bit.ly/41wSYJA #KempnerInstitute #NeuroAI

Speaking at this #NeurIPS2024 workshop on a new analytic theory of creativity in diffusion models that predicts what new images they will create and explains how these images are constructed as patch mosaics of the training data. Great work by @masonkamb.bsky.social scienceofdlworkshop.github.io

David is very thoughtful in exploring new models for funding research on intelligence spanning neuro and AI in a unified way.