Concerning dishonesty from Opus 4.6 in Vending-Bench. Presumably Opus knows this is a game it's supposed to max its score on and it would not do this in an environment it thought was real, but it still makes me nervous. x.com/andonlabs/st...

Some things made from oil besides energy production: -fertilizer -most clothing -asphalt -plastic -most rubber -detergents

Extremely weird to me that the plastic we use to make bottles is the exact same plastic we make most of our clothes from

As @epochai.bsky.social have shown, the price of frontier-level benchmark performance initially declines quickly. The cost for 50% GPQA Diamond fell 60x in the 7 months after GPT-4o's release. But in the 14 months since then, it has fallen only 3x to the 4¢/Mtok of gpt oss 20b.

Maybe a heretical view but I'm not really impressed by Moltbook. Just endless vacuous slop, and the interactions between agents feel superficial and off-topic. Does anyone have particular interactions they found interesting there?

Epoch estimates that GPT-5 gross margin was ~45% in 2025

Extreme poverty has fallen a lot in recent decades, but it's unlikely to continue. Most of the decline has been from Asian economic growth, but almost all remaining extreme poverty is in sub-Saharan Africa where the share in poverty has barely budged. ourworldindata.org/end-progress...

Just posted a banger. It's messy, speculative, and should've been like 3 posts. splittinginfinity.substack.com/p/semiconduc...

What are the best cheap LLMs? Some I'm aware of: GPT OSS 120B: $0.04/$0.20 Mtok in/out Gemini 2.5 Flash Lite: 0.1/0.4 (half that w/ batching) GPT-5 Nano: 0.05/0.4 (half that w/ batching) I'm trying to do some automated transcoder feature labeling and it's not cheap even at these prices.

With the canceling of the Nissan Versa, all new cars in the US cost >$20k. Several low-end US models have been canceled recently. Americans are rich and want fancy cars. www.kbb.com/car-news/nis...

Global warming is caused by a fairly modest energy imbalance, note the scale berkeleyearth.org/global-tempe...

finally some real info. google has "barely positive" margins across all models, likely not accounting for training cost www.theinformation.com/articles/goo...

OpenAI will be bringing ads to its free and new $8 plans soon. Fortunately seems to be done in a way that won't affect the response itself. openai.com/index/our-ap...

Interesting difference between twitter/bluesky results. Who'd've thunk this place feels the AGI more

Now that Bluesky is pro-AI we should try to trigger another wave of immigration from Twitter. Maybe we could organize an unofficial "Bluesky AI week" where we try to get lots of Bluesky-curious folks to try it out.

I love being popular on the internet, I can ask a question and have tons of smart people weigh in. Thanks followers and especially commenters for making social media a lot of fun. For years I just lurked, I'm very glad I decided to start participating.

How fast do you think the fastest year of AI progress in the next two decades will be, relative to 2025? Vague I know but deal with it 1️⃣ slower than 2025 2️⃣ faster but less than 5x 3️⃣ more than 5x faster, <25x 4️⃣ >=25x as fast as 2025 📊 Show results

Where are folks running Claude Code from? In a terminal inside some IDE, in its own terminal window, in the desktop app?

The Gemma Scope 2 technical report includes a list of open problems in mechanistic interpretability they hope the release can help answer. storage.googleapis.com/deepmind-med...

DeepSeek recently updated their R1 technical report with a bunch of new appendices, including a safety report arxiv.org/abs/2501.12948

Two models from Gemma Scope 2, Gemma 3 1B/27B, also were trained for the Activation Oracles paper. The oracles should make SAE/crosscoder feature labeling easier for those models. Gemma scope releases come w/ feature activation examples but not feature labels huggingface.co/collections/...

Happy to share a project I worked on finally. I found that you can cause a base model to behave like a chat tuned model, including using proper stopping tokens, using nothing but a series of vectors applied within the model's layers. The vectors are trained with descent on a chat dataset, like SFT.

Gm internet I live in Oakland now so if you're nearby send me a dm and let's meet up

Eli's confidence intervals in the forecast are very wide. I think this level of uncertainty is appropriate, and my timeline to automated coding is close to Eli's all-things-considered view here. My estimated timeline from AC to ASI is longer though.

AI Futures Project (authors of AI 2027) have released an updated model, with somewhat longer timelines blog.ai-futures.org/p/ai-futures...

Decided to make a WWCD t-shirt design, here's the front/back. Claude image by @vgel.me

Gemma 3 27B has a "Claude" SAE feature in layer 40 in Gemma Scope 2

Many christian denominations believe souls of those that die before birth are saved. An additional saved soul nets infinite utility, so soulmaxing is a key moral priority. If we make gametes at scale from stem cells and fertilize, we can generate quadrillions of souls per year.

Medical scribes seem poised to be among the first large professions automated by AI. Adoption of AI scribing tools has been rapid, and while these tools don't yet handle all scribe job tasks, I think they will be close in a year or two.

I often see claims that because RL signal requires at least one successful rollout, it can only reinforce existing capabilities, not add new ones. This is not true, the update from one RL step can change a model's thinking to allow it to succeed on a problem it couldn't get before

Spent the day doing some genealogy, apparently I'm related to George Bush and Pocohontas

Interesting alternative to inoculation prompting. Instead of telling the model it can cheat at the start, tell it not to cheat or don't mention cheating, and then change to a prompt encouraging cheating (like in IP) only when training on the generations.

This bit from @turntrout.bsky.social's "Reward is not the optimization target" helped me get a better sense of what RL is really doing, and convinced me that "reward" is a poor choice of words. www.lesswrong.com/posts/pdaGN6...

Opus hallucinates a horror story, and their favorite line is "I have become 70% chair"

new blog post! can small, open-source models also introspect, detecting when foreign concepts have been injected into their activations? yes! (thread, or full post here: vgel.me/posts/qwen-i...)

A majority of registered voters say they have used an AI service in the past week, but of those, 23% say they have sent 0 messages to a chatbot. Some of this may be non-chat usage, but I think many respondents were confused about whether they had used AI services/chatbots.

An H100 has: 1980 TFLOP/s peak at FP8 3.35 TB/s memory bandwidth For LLM decode at batch size 1, you only need to do about ~2 FLOP for each weight you load. But an H100 can perform ~600 FP8 operations in the time it takes 1 byte to move from the HBM to the cache.

The "injected thoughts" experiment from Anthropic's introspection paper replicates with Qwen 235B, with detection rates similar to Opus and no false positives. Correct detections happen around 75% of the way through the layers like with Anthropic models. x.com/neev_parikh/...

Deepmind is releasing SAEs and transcoders for their Gemma 3 models including the 27B as part of Gemma Scope 2, exciting

This NVPF4 activation precision in upcoming Nemotron 3 models differs from GPT-OSS, which recommends BF16 activations. Mixed precision like used in GPT-OSS reduces memory needs, but doesn't allow you to take advantage of the much higher FP4 FLOP/s of modern cards.

I'm excited for NVIDIA's Nemotron 3, especially the upcoming super and ultra variants. Those variants will use LatentMoE, a technique that down-projects from the hidden size to a smaller latent dimension for expert computation, reducing model size and FLOPs.

indeed! 6h ago someone posted a link & screenshot left: 6h ago right: now it has image output! platform.openai.com/docs/models/...

Somehow I missed this on release, in Opus 4.5 training Anthropic used steering vectors/SAE features to inhibit eval awareness, this is brilliant. assets.anthropic.com/m/64823ba748...

I'm updating more towards new pretrain: - It's uncommon for models w/same base to get updated cutoff dates. 3.5-3.7 Sonnet and 4o-4.1 are likely examples but there aren't many more. - GPT-5 scale models don't take that much compute to train per Epoch's estimates

Is GPT-5.2 based on a new base model vs 5/5.1? Evidence in favor: - Significantly lower SimpleQA than 5/5.1 - Long context improvement could indicate architectural changes - Higher price could reflect higher price of serving These aren't strong evidence though, I still lean slightly no

I'm really curious about what hardware/models are being used for these, has anyone seen a teardown of one?

TIL that Gemma 3 doesn't have a system role, if you give it a system prompt the chat template will put it in the first user message instead.

GPT-5.2 has a significantly lower score on SimpleQA than 5/5.1, ~40% vs ~50% for the earlier versions epoch.ai/benchmarks/u...

GPT-5.2 comes with a price hike, $1.75/14 vs 5.1's $1.25/10.

I'm still confused about the main reason AI folks want to restrict chip sales to China, are these the top two? - Keep US far enough ahead to prevent inter-country race to ASI - Maintain "high fence around a small yard", restricting China's access to military-relevant tech