If you're looking for the 60 minutes piece on CECOT that Bari Weiss canned to protect the President, you can find it here: archive.org/details/60-m...

Incredibly proud of my friend and colleague @storytracer.com. Two weeks ago he and his cofounders @sucho-org.bsky.social were honored for organizing a global network of volunteers to exfiltrate and back up endangered Ukrainian cultural heritage in the wake of the invasion by Russia.

Really great to see NVIDIA staking out a pro-open data position. This used to be common, if not the norm in AI, and the backing away from this level of transparency has done a lot of harm to the research community.

Compare the statement about the antisemitic terror attack in Australia by the Israeli Prime Minister with the one by Zohran Mamdani, and ask yourself who more truly cares about condemning antisemitism, as opposed to using it to promote unrelated politics.

"The incentives made me do it" is an excuse, not a justification. You can be better than that, and if you're not it's because you choose to not be.

It's a wrap on EvalEval in San Diego! A jam packed day of learning, making new friends, critically examining the field of evals, and walking away with renewed energy and new collaborations! We have a lot of announcements coming, but first: EvalEval will be back for #ACL2026!

In 2023-ish it was trendy to write papers trying to explain why scaling laws had power law structures. The papers I remember were pretty unconvincing. Did anything meaningful come of this work? What does the best work in this vein look like?

Put this person in jail. Put the person who drafted it in jail. Put the higher ups who covered it up in jail. This is a crime. If Kilmar had done the same they wouldn't hesitate to punish him. ICE is a criminal organization, not a law enforcement organization, and justice requires accountability.

Hyped to write "The models in this paper cost us 476,246.57 USD to train. I'm sorry you are sad we didn't redo all of our experiments on multiple independent training datasets. If you'd like to give us a million dollars we'd be happy to run the experiments you wish" in my response to a reviewer.

🚨 AI keeps scaling, but social impact evaluations aren’t–and the data proves it 🚨 Our new paper, 📎“Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations,” analyzes hundreds of evaluation reports and reveals major blind spots ‼️🧵 (1/7)

Our #NeurIPS2025 paper shows that even comparable monolingual tokenizers have different compression rates across languages. But by getting rid of whitespace tokenization and using a custom vocab size for each language, we can reduce token premiums. Preprint out now!

I feel like there are several blog posts or papers that put forth a research agenda of "making AI research a scientific field" or "advancing the science of AI" or something like that. I'm trouble finding them, does this ring a bell to anyone / does anyone have links to notable examples?

I want to print it out giant and put it everywhere

The new “teen safety” program from OpenAI repeats the same lies that companies and governments have been saying since the internet began. This won't achieve better online safety for kids, but it will suppress individual liberty and promote censorship. openai.com/index/buildi...

the *cato* institute says less than 10% of politically motivated terrorism is caused by leftists. the *cato* institute. more than two-thirds is from the far-right.

How can an imitative model like an LLM outperform the experts it is trained on? Our new COLM paper outlines three types of transcendence and shows that each one relies on a different aspect of data diversity. arxiv.org/abs/2508.17669

How did you learn to present code? Are there resources that you recommend using to help teach people?

On Thursday we will be joining the #UCLA Latino Policy & Politics Institute for their demo of the newly updated Latino Data Hub (LDH), a public bilingual data platform built to democratize access to critical data about #Latino communities across the country. Register here:

Here are a couple of slides that I presented yesterday at #aitechgov about open-weight model risk management.

Thanks to @stellaathena.bsky.social for chatting with me about Deep Ignorance: the new paper/project from Eleuther AI and the UK AISI. Bottom line: Worried AI could teach people to build bioweapons? Don’t teach it how fortune.com/2025/08/14/w...

“Your driver's license contains a ton of somewhat immutable information about you” like your name, address, DOB, and face, EFF’s Lisa Femia told the @thetennesean.bsky.social. It's not like a credit card number that can be replaced if it's leaked.

Are you afraid of LLMs teaching people how to build bioweapons? Have you tried just... not teaching LLMs about bioweapons? @eleutherai.bsky.social‬ and the UK AISI joined forces to see what would happen, pretraining three 6.9B models for 500B tokens and producing 15 total models to study

Amazing work by @lyndamk.bsky.social and @datarescueproject.org to preserve history in the face of fascism. #SaveOurSigns

"It's unclear if this research matters because real users speak English and Chinese" has got to be up there for worst dismissive takes about how multilingual doesn't matter.

I forgot that Scholar does alerts not just for keywords but also for specific people so I made alerts for all my advisees and now I get an email when they have a paper out maybe folks already do this and I'm late to the game but honestly those alerts feel great, esp when it's a long-gone advisee

It really annoys me how people have made prominent careers out of writing a paper with a scary headline about AI risks that would have produced a non-scary headline with a minor tweak to the set-up and/or probably isn't a meaningfully real phenomenon in the first place.

Congrats Moonshot on making a "modified X" license that's still open source. This is also making me wonder about the list of models to hold the title "most powerful open source LLM in the world." GPT-2 > GPT-Neo > GPT-J > FairSeq Dense > GPT-NeoX-20B > MPT-7B > Falcon-40B > ??? > DeepSeek-R1

At #ICML2025 and a fan of EleutherAI? Come find me for laptop stickers!

Another banger from one of the most important interp researchers in the world.

Stop by our discover server tomorrow, Friday June 27th, to hear about @catherinearnett.bsky.social's work!

This continual level of ignorance (although I am tempted to assume bad faith or its rhetorical equivalent) is not only dangerous, but undermines work towards very real problems it seems to have originated against (silicon valley hype, corporate control, environmental concerns, the rising tide of

We got another mention! We have a lot of these @noaa.gov datasets backed up now. Be sure to check out our tracker: baserow.datarescueproject.org/public/grid/... Scientists scramble to save threatened federal research databases pubs.aip.org/physicstoday...

A bunch of papers suggest that if X and Y are independent tasks, we might expect to see "emergent" behavior on "X and Y" or some task that requires first X and then Y. I'm really surprised I can't find any papers that dig into this; it's usually a side comment. Do you know any?

Several other groups have put out openly licensed dataset recently, why is ours better? Ablation studies show trained on Common Pile v0.1 outperform them, matching the performance of models trained on the original Pile and OSCAR, though still falling short of FineWeb

Our pretrained models, Comma v0.1-1T and -2T perform comparably to leading models trained in the same regime. These plots also include Qwen as a SOTA 8B reference, though it saw 36T tokens

We put a lot of work into our metadata, such as having two rounds of manually validating the ToS of websites in Common Crawl, manually identifying trustworthy YouTube channels, and leveraging work by the BigCode Project and @softwareheritage.org to build the openly licensed subset of StackV2.

The project of open science for machine learning only works if we are able to distribute the training data. Openly licensed data lets us do that, under mild conditions. We make sure to provide document-level metadata for authorship, licensing information, links back to the originals, and more.

The Common Pile comprises text from 30 distinct sources, covering a wide variety of domains including research papers, code, books, educational materials, audio transcripts, governmental text, and more. Some of this text is commonplace in AI, but a lot of it is pretty new.

Can you train a performant language model using only openly licensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1 & 2

Is there a way to search Google Scholar or Semantic Scholar and get a list of all papers where at least one author has a specific affiliation?

Reading the ToS of a thousand websites is exhausting. Please send coffee and hugs.

The Trump administration is continuing its illegal attacks on the scientific establishment by withholding funding and unilaterally ending contractual obligations. It's not about science or policy or national security. It's about the wholesale destruction of academia via any lever they can reach.

People keep plugging AI "Co-Scientists," so what happens when you ask them to do an important task like finding errors in papers? We built SPOT, a dataset of STEM manuscripts across 10 fields annotated with real errors to find out. (tl;dr not even close to usable) #NLProc arxiv.org/abs/2505.11855

Wow. We've had a number of new followers recently. Welcome to the wild and crazy world that is the rescuing of federal data for the public good. Check us out: www.datarescueproject.org We also like emojis. ❤️🛟

No, I didn't fail to win any NSF grants last year. I made the strategic decision to seek funding elsewhere to protect the financial solvency of my non-profit.

I feel so bad for the immigrant scientists in the US who won’t see a conference in the next 4 years because we’re a scientific pariah state. Especially the PhD students, but also postdocs who never got to meet the global community during COVID.

Today, at 11am ET, @storytracer.org will be giving a live demo on the @mozilla.ai Discord showcasing two Blueprints for creating open datasets: audio transcription using self-hosted Whisper models and document conversion using Docling. Join the event here: discord.com/invite/4jtc8...

After a short era in which people questioned the value of academia in ML, its value is more obvious than ever. Big labs stopped publishing the minute commercial incentives showed up and are relentlessly focused on a singular vision of scaling. Academia is a meaningful complement, bringing... 1/2

It was great to speak to @chronphilanthropy.bsky.social about AI research and how it's challenging yet necessary for non-profits to work in the LLM space. Many think we should leave the field to companies, but non-profits have different values and goals and those are important. shorturl.at/8IlmF

Proud to be at the AI Action Summit representing @eleutherai.bsky.social and the open source community. The focus on AI for the public good is exciting! DM me or @aviya.bsky.social to talk about centering openness, transparency, and public good in the AI ecosystem.