might be time

I just learned that John McCallum, who maintained an excellent database of storage costs over time, has died. In his final update, he said "I must pass the work of keeping the data updated to the next generation". Has anyone yet picked up the torch, and if not, any advice on how I might?

Anthropic welfare researcher Kyle Fish suggested a similar idea on his recent 80000 hours appearance

Are there LLMs that have been post-trained to be something very different from the assistant persona most LLMs are, but still designed to interact with humans, not like a classifier or something?

Qwen3-Next models are now up on Huggingface: huggingface.co/collections/...

Qwen3-Next-80B-A3B will probably have 10 routed experts active out of 512 based on these defaults in the Qwen3Next documentation, this matches the 1:50 ratio claimed. Config also mentions linear attention layers, MLP-only layers, and a shared expert huggingface.co/docs/transfo...

From 2015-2024, wage growth at the low end exceeded growth at the high end. Sadly it looks like that's no longer the case. www.nber.org/system/files...

Feeling sick to my stomach on hearing that Charlie Kirk has died. Political violence like this makes us all worse off.

I feel like this has potential as a political topic reaction image

Interesting bit from the first couple min: Training run compute for frontier models has been growing ~5x/yr, but is likely already slowing: - Longer training runs have accounted for 1.4x of growth, and unlikely to continue - Larger data center buildouts are taking longer

Qwen3-Max-Instruct-Preview is now out, and performs well on benchmarks. But with closed weights and a price of $1.2 input/$6 output, I'm just not that excited. Realizing that a lot of my excitement when new models are released is getting to see the architecture details. openrouter.ai/qwen/qwen3-max

I've been wishing for a deep dive into efficient MoE inference for a while, today my wish comes true. One big takeaway seems to be significant economies of scale, the batch sizes here are enormous.

One important point this piece by Birch and Suleyman's SCAI piece get right is that many people will think the AI models they use are conscious simply because they act human-like. Regardless of actual consciousness, being "friend-shaped" will likely shape public view.

I agree with the two-issue framing of this piece, but I disagree strongly with the doctor's note analogy and the comments on relation R, and think they incorrectly describe LLM inference. Here's where I disagree, copied from my response on Twitter:

Digit carries cinderblocks around with our new RL whole body control policy

There's an AMA happening right now with the Z.AI folks behind the GLM models over on the LocalLLaMA subreddit

Our modern diet provides far too low a ratio of omega-3 to omega-6 fatty acids, with few good omega-3 sources besides fish. But this Australian company made a GMO canola with lots of the long-chain omega-3 FAs our body needs. These folks are heroes, god bless GMOs and God bless Australia.

Does anyone know of attempts to estimate the amount of AI compute available over the next five years or so? Seems it should be possible to get a loose approximation based on TSMC's expected production of CoWoS wafers at each process node, right?

Bluesky is now unavailable in Mississippi due to the state's law requiring age (& implicitly identity) verification on all social media. The supreme court recently declined to put a hold on the law but said it likely violates 1A. I sure love our Bill of Rights, ty founding fathers!

I was looking back through my old pics today and came across a few videos I took during the 2020 Seattle protests, in retrospect I wish I took many more. Here's some of what I got: May 30, shortly before imposition of 5 p.m. curfew, protesters flood onto I-5

Been meaning to make this for a while

When Gemini 2.0 Flash Preview launched with native image generation shortly followed by 4o, I thought this multimodality would become the norm. But newer models from Google/OpenAI have not had this feature, and no other labs have released models with native image out. What gives?

When searching, character.ai fetches its tag taxonomy as a json, these tags are not shown directly to end users. Some questionable personalities in there IMO.

The top models on OpenRouter suggest very high input:output token ratios, but large portions of these input tokens are probably cache hits, in my Cursor use cached input tokens dominate non-cached. Is there any data on relative input(cached):input(non-cached):output token use?

character.ai is the largest platform for character-based AI, so I figured I should scrape them in addition to Meta. Takeaways: -So much anime, >1/2 of top 100 -Many are scenario rather than single char -More male chars than female -As w/ Meta some have romantic themes, but small %

If you try really hard you can fit a whole image in the alt text on Bluesky. Take the string from the alt text here and paste it into the field on this webpage, and you'll see a photo I took! www.ipvoid.com/base64-to-im...

I made estimates of revenue from OpenRouter's top models over the last week from token counts/typical prices. Of $3.2M in the last week ($165M annualized), 75% is attributable to Sonnet. Output token share is surprisingly low. These models account for ~80% of tokens served, & probably 90% of revenue

I made an attempt at compiling popular Meta AI characters w/ some scraping and searching. Major themes from and beyond this top list: -Indian women, seems India is leading the AI gf race -Astrology, lots w/ Indian themes but not all -Anime -Most seem more social than romantic

@julia.cool finetuned a version of Gemma 3 270M on her tweets and has posted some curated outputs, they're quite cute (as small model outputs tend to be). I didn't quite grasp at first that finetuning is the whole point of this model, it's too small for general use but can be tuned for specific uses

What's the best case you've seen for strong emergence? To me it looks like we can explain any emergent feature as a pattern in an underlying fundamental one, but I should at least try to understand where folks who think otherwise are coming from.

Hare are some responses from the new Gemma 3 270M model. I'm using the Q4_0 quantization, the bf16 may be a bit less dumb. I'm also not using a system prompt, which may hinder coherence. First off, "what is it like to be a bat?"

Google just released a 270M parameter Gemma model. As a tiny model lover I'm excited. Models in this size class are usually barely coherent, I'll give it a try today to see how this does. developers.googleblog.com/en/introduci...

W3Techs has tracked the image formats used across the web since 2014, counting a format as used if it shows up anywhere on the site. SVG and WebP are both on the rise. w3techs.com/technologies...

xAI co-founder Igor Babuschkin is leaving, in part to support AI safety work: x.com/ibab/status/...

Short timelines people today have noting on my grandpa, who in 1958 predicted "Within the next two years the computer will be usable for industrial problems such as, "How should I vary my factory production from hour to hour to make the biggest profits?"

TextQuests is a new LLM benchmark measuring performance on a suite of text-based interactive fiction games, published today. Lots of room to grow on this one, GPT-5 scored the best completing 5/25 games with official game hints and 0 without. huggingface.co/blog/textque...

Correlation doesn't imply causation, but it is correlated with causation.

"AGI timeline" is way too vague. Here are some weakly related timelines I like that are much more quantifiable: - Time until top 3 AI companies earn 1T in one year (or some other # of firms/revenue) - Timeline to reach 90% on VideoGameBench - Time to AI-powered robot doing sprint triathlon in 2h

I don't know whether human-AI friendships/relationships will be good or bad on net, and I don't think we'll know for a while (I think the jury is still out on social media decades later). I think one important factor will be the degree to which they crowd out connection with other humans.

A very interesting (and very thorough) case for Alzheimer’s as (of all things) a disease of lithium deficiency. This one’s worth taking seriously, I believe:

Now available for ~50% less, though tok/s for Chutes is lower. I wonder how Chutes consistently manages to have significantly lower prices than other OpenRouter providers.

This tweet was partially wrong. Unless a black hole is large enough to not dissipate in a trillion years, it will have a high black body temp and will emit electrons and heavier particles. More importantly at any size it will emit neutrinos, which are impractical to collect.

OpenAI is bringing 4o back to ChatGPT for Plus users after widespread dissatisfaction with the change on /r/ChatGPT www.reddit.com/r/ChatGPT/co...

OpenAI is surprisingly transparent about GPT-5 usage limits: help.openai.com/en/articles/...

GPT-5 API prices are largely similar to the models they're replacing but input tokens are cheaper and cached input tokens much cheaper. The lower input/output price ratio could indicate that these are sparser MoEs than before since MoE sparsity lowers input token cost more than output.

Plenty of people listen to podcasts at >1x speed but the real big brain move is listening to music at >1x.

Once the Sun runs out of hydrogen/helium in 5B years it will collapse to a white dwarf, a dense hot ball that will slowly cool to a black dwarf over 10^15+ years. But then what? We think protons probably decay on timescales of 10^35+ years, if so it will slowly shrink to nothing.

Uranium fission converts ~0.1% of its mass into energy, and hydrogen fusion converts ~0.7%. Pitiful. But accretion disks of black holes can convert up to 40% by extracting gravitational potential energy, and creating tiny black holes and collecting Hawking radiation converts 100%.

The lowest price for GPT OSS 120B on OpenRouter is currently $0.15/0.6 per million input/output tokens, but I expect that cost to fall. Deepinfra offers an FP8 version of Llama 3.3 70B for $0.038/0.12, if anything serving OSS 120B should be cheaper due to the MoE architecture.

When an open source model is released one thing I like to is take a look at its config.json on HuggingFace. Some initial observations on the new OpenAI models: - Both have 4 active experts, as opposed to the 8 we've seen in most recent releases - 128 total experts for the 120B model, so it has ...