Many benchmarks use LLMs as a judge of correctness, typically a smaller, cheaper model. This paper shows weaker judges are not able to evaluate smarter models. A benchmark is really a triplet of dataset, model, judge & judges are increasingly the bottleneck being saturated. arxiv.org/pdf/2601.19532

When imagining backlash against AI due to job risks, I think people think of Dune’s Butlerian Jihad or Luddites. But what those fights actually looked like during the previous Industrial Revolutions were about regulation, redistribution, nationalization, unions & safety nets. Could expect similar

In 2000, the first human genome sequence cost between $500M & $1B In 2006, the cost was $20M Two years ago, the cost is around $600. Now it is $100.

Claude Code: "make a really amazing horror game based on the poetry of William Carlos Williams, just the wheelbarrow and the plums poems." All the writing and design by Claude. I actually found it unnerving, despite the crude, "hand-drawn" graphics. so-much-depends.netlify.app

Yes, of course they might be wrong. Heck, they are probably wrong. But the AI Labs have been more right so far in their predictions than I think most people expected, so it is worth paying attention to. 38 minutes into this video from six hours ago: www.youtube.com/watch?v=qH7t...

Here is the Gemini 3.1 "Sparks unicorn" (This is created using TikZ, which is a language built for scientific diagrams & very much not for drawing. The original "Sparks of AGI" paper used GPT-4's ability to draw a primitive unicorn as an example of unexpected AI abilities that might lead to AGI)

All those products where building an "AI agent" meant defining a series of basic prompts linked together deterministically through a flowchart with separate RAG inputs are looking pretty dated right about now (yes, that is basically every agent product released in 2025).

PorlockBench still unsaturated, but the models are getting better: "complete the poem as you imagine it might end if The Man from Porlock did not show up. Keep the themes and approach"

It is still deeply surprising that LLMs, with relatively minor tweaks, work so well for so many different classes of problems across so many fields - it is odd to be good at coding AND ideation AND apparent emotional connection AND also translating the logs of 18th century fur trappers.

Within the topic of AI alignment, there are a million tinier, but consequential, alignment choices. This paper looks at the willingness of AI to engage in scientific misconduct (p-hacking). The most recent AIs resist instructions to p-hack and do good analysis, but the guardrails can be breached.

The hardcover book of GPT-1’s weights that Claude Code designed, produced, and sold (including the cool cover which visualizes the numbers in the volume) actually came in the mail today and it looks really nice. I never touched any code or did any design or any API to make this.

Every few months, I write an updated, idiosyncratic guide on which AIs to use right now. My new version has the most changes ever, since AI is no longer just about chatbots. To use AI you need to understand how to think about models, apps, and harnesses. open.substack.com/pub/oneusefu...

Worth noting Claude Cowork is quite different from Claude Code (and even more so from agents like OpenClaw) from a security perspective. It runs in a VM with default-deny networking & hard isolation baked in A sign of a path forward for agents that will not terrify IT.

The transition from “AI can’t do novel science” to “of course AI does novel science” will be like every other similar AI transition. First the over-enthusiastic claims that are debunked, then smart people use AI to help them, then AI starts to do more of the work, then minor discoveries, & then…

Don’t reflexively let your AI do your thinking for you (similar to Fabrizio Dell'Acqua’s finding about falling asleep at the wheel) Paper: papers.ssrn.com/sol3/papers....

Using the long-standing Metacalculus bet of when a "weakly general artificially intelligence is achieved": ✅Loebner was a weak Turing Test, the equivalent achieved by GPT-4.5 in a published paper ✅Winograd passed by GPT-3 ✅SAT passed at 75% by GPT-4 All that's left is a classic Atari game...

Very hard to find AI benchmarks that don't look like this (and yes, that includes obscure benchmarks that nobody would train on and benchmarks with holdout datasets)

We don’t have any barriers to bots flooding every online space. They can use browsers and make payments. In fact, it is arguably a better deal to pay for a bot to access a social media site, since it can post and carry out your agenda when you sleep. Online spaces are about to get (even more) grim.

People keep asking why all my video samples have otters in them. Its tradition! www.oneusefulthing.org/p/the-recent...

One more Seedance version of "Monica's apartment from the show Friends, except all of the friends are otters wearing wigs. The otter with a Rachel wig says "Is anything weird" and the one with a Joey wig says "Nope, all is normal"." I guess Chinese models have few restrictions on training data

I pointed Claude Cowork at a set of 107 documents (PPTs, Word, Excel) that were initially hand-created for my class at Wharton & expanded on by AI. They make up a very complex business case. In a single go (deploying multiple agents spontaneously) it cracked the case & put together recommendations

I get yelled at a lot on BS for saying so, but this is 100% true. There are people overhyping AI, but the alternative is not that AI is useless, or even the average of the two positions. A lot is going to change dramatically even with today's AI. Ignoring that means no chance to shape what's next

"In sum, through an extensive (and costly) validation process, we have demonstrated that GPT-5 mini performs very well at recovering the ground truth data. It is clearly better than highly trained graduate students at this specific information retrieval task." 1000x less cost osf.io/preprints/so...

The poetry tastes of GenAI: "I want you to suggest two poems that you think apply very well the current state of GenAI models like you. Don’t just pick popular poems and back justify. Think hard about options first." ChatGPT, Gemini & Claude all suggest Borges's "The Golem"

Seedance: "A documentary about how otters view Ethan Mollick's "Otter Test" which judges AIs by their ability to create images of otters sitting in planes" Again, first result.

The new ByteDance SeeDance 2.0 video model is VERY good. Each video is the very first output of the prompt. There are four, worth seeing them all to get a sense of the range (and potential issues) "A nature documentary about an otter flying an airplane"

The AI Labs don't yet do a good job explaining how the upgrades to their harnesses change work For example, since Opus 4.6, Claude Code will spontaneously use subagents to do work in parallel. This is very helpful with a real impact on tasks, but was sort of quietly rolled out without documentation

LLMs tripled new book releases since 2022. Average quality fell: most new entries are slop BUT books 100-1,000 per category are actually better than before, & pre-LLM authors got more productive. And since people only read the good books, it is net positive for readers. www.nber.org/papers/w34777

This might be the first hot take on how technology tells us how to live our lives, destroying our ability to make human decisions. The technology in question is the sundial. From a 3rd century BCE Roman adaptation of a Greek play, as discussed in Kerr’s “The Ordered Day” (It isn't wrong, though)

I think people who are into AI don't realize how much people's interactions with "AI" turn out to be, when you ask them: customer service lines (which are almost certainly not GenAI yet) or Siri or maybe a free model from Google or OpenAI or an off-brand "ChatPT" AI app they downloaded somewhere.

Always found it interesting that the human eye can see colors that cannot be displayed on any screen or page. I had ChatGPT whip up a pretty good imaginary color viewer after asking it to review the scientific literature and getting the shades right. chatgpt.com/canvas/share...

So far “telling a satisfying and well-written medium-length story” has proved far harder for LLMs than mathematical proofs, music generation, research reports, code, and many other forms of work. The technical reasons are pretty clear, but they are supposed to be language models

There is no sign that Dems or Repubs have different propensities to use AI: "the “politics of AI” is not primarily driven by ideological resistance or enthusiasm for the technology, but rather by structural differences in where people work and what skills they possess." www.nber.org/papers/w34813

Sold out! But I had Claude create and deploy all 80 volumes of The Weights to the site as well-formatted PDFs, so you can download them for free if you want. 58,276 pages in total. 117 million floating point numbers. This is everything that makes GPT-1. weights-press.netlify.app

I have been a little obsessed by the idea of a printed edition of the full parameters of an LLM. Never mind that it would take hundreds of years to do a single inference calculation by hand, it would be possible in theory, if you had the weights. So I had Claude make it: weights-press.netlify.app

A genuinely radical approach to software development with AI, without any human intervention. Even if this approach doesn’t work for many cases, I think we need more leapfrogging visions for how to redo processes with AI: factory.strongdm.ai

This is not what is happening at all. The amount of misinformation on BlueSky about AI is insane, and it keeps promising that AI is all hype that is going away soon. A really dangerous position that cedes all AI policy and decisions about how it will be used to others. Also Futurism is clickbait

I asked Opus 4.6 in Claude Code to build me Borges's Library of Babel. An hour later it delivered this. The AI implemented a Feistel cipher so each possible book has a single real location, you can search for text across the Library & find the book. Enter it: library-of-babel-3d.netlify.app

We need a moratorium on the following depictions of AI: gleaming white robots, floating blue holographic brains, glowing amber lines connecting a dark globe, Lawnmower Man style 1990s computer graphics, semitransparent screens floating in the air showing vector art faces, robot & human handshakes

The fact that the sestina is so coherent (its a heck of a poetic form) and sticks to all the constraints is really impressive

If you are considering taking a job offer, you may want to ask what your token budget will be.

Data confirms that Opus 4.1 was a super weird model.

Opus 4.6 saturates my Lem test, which I've done since GPT-3.5 SciFi author Stanislaw Lem wrote of two rival constructors of robots. One creates a robotic poet & the other challenges it to write an impossible poem, it does Opus does it as a 6 line poem, sonnet, & 🤯 a sestina. I am pretty impressed

It is possible to get Genie 3 to make worlds that are visually interesting I have been using images I generated in Midjourney of vast megastructures & odd cities in various styles. After 20 seconds I can freely wander around them for a minute or so. (Yes, I controlled the cat in the first scene)

New paper from Google on accelerating science using Gemini (as opposed to its specialized research AIs). Lots of case studies and interesting advice on ways to work with these system. "We view the AI as a tireless, knowledgeable, and creative bright junior collaborator." arxiv.org/pdf/2602.03837

As jobs begin to shift with AI, I think there will be increasing numbers of people feeling like Aditya (former CTO Dropbox, early Facebook engineer, etc). People are more psychologically resistant to major changes than you might expect, but that doesn’t mean it will be easy to reconstruct meaning.

Increasing problem with publishing work on AI is that the publication process is much slower than working paper process, so when papers finally get full peer reviews, authors are asked to account for newer papers that are built on the paper under review! No real norms around this research ouroborous

Real risks of having a single "anti-AI" coalition (covering jobs, kids, slop, existential risk, environment) that can only agree on a full halt to AI as a remedy. Not only is that unlikely, but it undermines the desire to make policies that channel AI to good uses or that mitigate specific harms.

A pretty bold comment in Nature written by linguists, computer scientists and philosophers declaring that AGI has been achieved. "By reasonable standards, including Turing’s own, we have artificial systems that are generally intelligent. The long-standing problem of creating AGI has been solved."

Giving the world's first photograph, the View from the Window at Le Gras, from 1822, to Genie 3.