Employing mechanistic interpretability to study how models learn, not just where they end up 2 papers find: There are phase transitions where features emerge and stay throughout learning 🤖📈🧠 alphaxiv.org/pdf/2509.17196 @amuuueller.bsky.social @abosselut.bsky.social alphaxiv.org/abs/2509.05291

Helpfulness is what we are after, and we test it by asking humans for preferences, or reward models. and they fail😆 They show that humans are bad at predicting what is helpful, so are reward models (all close to chance). Reward models don't even predict what helps LLMs RL🤔 🤖📈🧠 #AI #LLM

Good luck with the @iclr_conf writing Know anyone who needs tips? Want a graph checklist? Know any good tips you wanna add? The writing guide: docs.google.com/document/d/1...

The most expensive part of training is the data, not the compute Nikhil Kandpal & Colin Raffel calculate a really low bar for how much it would cost to produce LLM training data with 3.8$\h Well, several scales more than the compute. Luckily (?), companies don't pay for the data 🤖📈🧠

Anyone willing to make an emergency review for babyLM?

A dataset of ancient Chinese writings to study with LLMs, including 170K sentences for pretraining With 10K words, mapping to modern word (when applicable) There are so many fascinating questions out there www.arxiv.org/abs/2508.15791

Why is it hard to predict downstream scores from pretraining? For many reasons such as domain, mismatch between current abilities and what post training unfolds, "emergence" etc. A big factor is that next token prediction != choice comparison != accuracy www.alphaxiv.org/abs/2406.04391

The right path is unclear; maybe it doesn't even exist. Still, LLMs appear to consistently follow the values of secular\rational people who strive for self-expression (sounds like me😅) To show it they collect and release 200K human-model chats+feedback, 5 languages and 21 LLMs 🤖📈🧠

Pressing matters in evaluation from GEMs talk Remember, exciting questions drive science, exciting answers follow. Setting the right goal may make all their sota chasing worthwhile. Make an insightful dataset, lead by evaluation 🤖📈🧠

The talk for our work on Retrospective Learning from Interactions, which will be in ACL (once I figure out how to squeeze it shorter) Gist: autonomous post-training from conversational signals for LLM bootstrapping ... look ma, no annotations! no hand-holding! 🙌📈🚀 www.youtube.com/watch?v=qW8S...

ICML proposes we use sophisticated jailbreaks in our papers? Ones that trick the reviewers but do not raise our scores? Proposals?

Can LLMs learn social skills by playing games? A blogpost on human-model interaction, games, training and testing LLMs research.ibm.com/blog/LLM-soc... 🤖📈🧠

We don't understand loss spikes, that's clear. We've learned recently that data deterministically makes spikes regardless of optimizer. What did we see when we stopped pretraining, and then continue? A huge spike and never a recovery? Why? Apparently the momentum matters, a lot. 🤖📈🧠

Eu calls for evaluator experts to counsel about how to evaluate models (can't belong to a training company, smart) digital digital-strategy.ec.europa.eu/en/news/comm... #safety #eticalAI #AI 🤖📈🧠 Tag people who should know or should pass the knowledge

The ML\LLM AI world is overcrowded✅ and it's hard to find majestic new projects❌ Here is (updating) list of thoughts, premature ideas, and research directions that I cannot pursue alone, but I believe can make radical changes Please fight me, discuss, and ask Disrrruption, aye?🏴‍☠️ 🤖📈🧠

Surprisingly good tokenization workshop, resurfaced thoughts:🧠📈🤖 Why isn't tokenization learned? Can we do an evolutionary algorithm, or train a tokenization scheme on the pretrained or meta learn on something fast-pretrain (loss at the beginning of training?) Let's discuss👇

Do models really understand humans (ToM)? Can your model do it? Beat them at the neurips Competition. 🤖📈🧠

How do you test a new model? What's super challenging for models? The @kaggle.com team are starting a new way to crowd evaluate models, create your own benchmarks.🚀 (i'll tell you mine🧵 if you tell me yours) 🤖📈🧠 kaggle-task-generator-564531943159.us-west1.run.app

Models self-teach slightly more complex skills than the teacher knows And they can do so again and again until it is not just slightly. My questions:🧵

Teaching ChatGPT to tell everyone how handsome I am

Imagine a hacker causing ChatGPT to genuinely believe the stock market is crashing. No breach, no code, all an attacker needs is a malicious prompt and clicking 👍to change the model's weights. Our research paper demonstrates a vulnerability in LLMs RLHF mechanism: arxiv.org/abs/2507.02850 📈🤖🧠

The Swedish National Archives shared 1.2m images of handwritten documents from 1600-1900+ In addition, they share a model for writing to text Such marvelous data opportunities, kudos 📈🤖🧠 medium.com/@linnea.karl... Example: sok.riksarkivet.se/om-soktjanst... huggingface.co/datasets/Rik...

For the past ~2 months we have been working on training reasoning models on TextArena.ai games. Soon, a paper introducing what we think is a promising new paradigm (+ scaling and analysis). However, to get more feedback on the structure and implementation, we want to open-source the code now.

"Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning." arxiv.org/abs/2506.08872

🚀 Technical practitioners & grads — join to build an LLM evaluation hub! Infra Goals: 🔧 Share evaluation outputs & params 📊 Query results across experiments Perfect for 🧰 hands-on folks ready to build tools the whole community can use Join the EvalEval Coalition here 👇 forms.gle/6fEmrqJkxidy...

When Claude, Gemini, o3 battle each other for world domination in Diplomacy: DeepSeek turned warmongering tyrant. Claude wouldn't lie—everyone exploited it ruthlessly. o3 orchestrated a secret coalition, backstabbed every ally, and won. tx textArena.ai and friends every.to/p/diplomacy 🧠🤖 #AI #LLM

Remember Toyota scandal, where cars just "better behaved" when tested? If someone tested you, wouldn't you? And if you tested LLMs? Finding: LLMs can tell when they are evaluated🧠 We can only wait to see how they act on it alphaxiv.org/pdf/2505.23836 📈🧠🤖

Gradient norms rise at the end of training, and a new theory connects that to the learning loss when using weight decay. AdamC and SGDC(orrected) multiply the weight decay by the learning rate and fix it. alphaxiv.org/pdf/2506.02285 📈🤖🧠

Want people to interact with you? 💬 Open your DMs! Go to your Chats tab → tap the settings icon ⚙️ (Don’t go to Settings directly — that won’t work 😅)

They don't only "not consistently boost" they underperform :-(

Training LLMs on many games can generalize and teaches reasoning that holds in new environments. However, weighting the games is complicated, so merging (my beloved fusing in the title) is used. 📈🤖🧠 alphaxiv.org/pdf/2505.16401

I am glad that you found our paper entertaining! This is a great point for my follow-up thread on the implications of concept incongruence. Our main goal is to raise awareness and provide clarity around concept incongruence.

On the one hand, concept incongruence is very common. In role-play, “death” is our most vivid test case, yet the same tension appears any time a model must juggle incompatible constraints. This also shows up in political discussions, alignment (helpful vs. harmless), and essentially everywhere.

Highly entertaining paper and writeup, but does it really matter? Is it important that models can't abstain on counterfactuals? Or that the leak information?

I don't fully 100% agree/conform with this, but it's a good list!

Don't play games, create them

The cognitive side of babyLMs is up for the reading

Did you remember all the things to do before sharing a figure? Remove "_" from the labels save as pdf ... A lot of deeper tips to writing are in the shared document, but somehow this practical list was always the most popular. Good luck in #neurips docs.google.com/document/d/1... 🧠🤖📈 #LLM #AI

Has anyone written anything about *scraping and text processing* for internet pretraining data? Practical details, which tools are used, which webpage elements are considered, how HTML to text conversion is done? (I know about work on quality filters, relevant but not quite what I'm looking for)

Close your books, test time! The evaluation pipelines are out, baselines are released & the challenge is on There is still time to join and We are excited to learn from you on pretraining and human-model gaps *Don't forget to fastEval on checkpoints github.com/babylm/evalu... 📈🤖🧠 #AI #LLMS

Base Models Beat Aligned Models at Randomness and Creativity. Peter West & @cgpotts.bsky.social tell us that alignment doesn't only extract abilities hidden in the pretraining, it also hides other abilities: 📈🤖🧠 #LLM #AI #creativity

So far the best experience in getting knowledge out of bluesky! Highly recommended bsky.app/profile/did:... T.h @nsaphra.bsky.social Feilling with goods @tedunderwood.me

New preprint from @lauraknelson.bsky.social, @mattwilkens.bsky.social, and myself tests different ways of simulating the past with LLMs. We don't fully answer the title question here—just show that simple strategies based on prompting and fine-tuning are insufficient. +

We identify 20 dominant metaphors—ranging from "friend" to "god" to "thief"—and how their prevalence is shifting over time: warm, human-like metaphors (friend, assistant) are rising while mechanical ones that signal competence (computer, search engine) are declining.

Do different people really want different responses from LLMs? During ICLR we discussed that it is often not trivial, and except for clear clashes of morality and narratives. It is not obvious whether we force our wishful thoughts about different cultures on those needs. In a paper 🤖📈🧠 #ai #llm

Sparse attention is one of the most promising strategies to unlock long-context processing and long-generation reasoning in LLMs. We performed the most comprehensive study on training-free sparse attention to date. Here is what we found:

Here's what's on my mind😵‍💫 on yours as well? Talk to me at @iclr-conf.bsky.social or in general: Open feedback sharing Feedback loops Interactivity Multilinguality and multiculturalism Collaborative training (merging) Pretraining in Academia Evaluation 🧵 #LLMs #AI #openscience 📈🤖

TextArena is live on arXiv❗ We present a benchmark of 57+ competitive text-based games to evaluate&train LLMs including negotiation, deception, theory of mind... Multiplayer support Human-vs-models Model-vs-model Perfect for social interaction, Multi-Agent, multi-turn reasoning and Planning 🤖📈

How should the humanities leverage LLMs? ▶️Domain-specific pretraining! Pretraining models can be a research tool, it's cheaper than LoRA, and allows studying 💠grammatical change 💠emergent word senses 💠who knows what more… Train on your data with our pipeline or use ours! #AI #LLM 🤖📈

How should the humanities leverage LLMs? ▶️Domain-specific pretraining! Pretraining models can be a research tool, it's cheaper than LoRA, and allows studying 💠grammatical change 💠emergent word senses 💠who knows what more… Train on your data with our pipeline or use ours! #AI #LLM 🤖📈