so Bolts of Manic Inspiration from Athena are to be deeply, deeply distrusted but like, I do kinda think I basically figured out why LLM language starts getting weird in just the last two hours

me: what do you think of this argument Claude Claude: seems sus me: so that argument was from Benoit Mandelbrot Claude: never mind, it's brilliant

"The Amazing Digital Circus" honestly has more of the original, 2012 LessWrong AI-doom flavor in it than does Pantheon

In which I consider six conceivable causes of LLM unintelligibility I just really enjoyed writing this, fun topic that seems really unexplored 1a3orn.com/sub/2025-10-...

LLM slop about new fundamental understandings of the world, but imagined in a world where Catholicism was somehow the dominant meme, so all the metaphors are drawn from there rather than "quantum mechanics", recursive stuff, etc

William Blake: "The road of excess leads to the palace of wisdom." Me, after I've listened to my autoconstructed Youtube music playlist for so long it's wandered out of my usual repetitive basin: "So true."

starting an anime episode titled "prologue" and we meet a mom, in a risky job, with a small child, in the first two minutes yeah there's never been anyone more doomed surely

ok is it nuts for me to think you can tell the political affiliation of a show simply from the art style and composition of certain shots

if there's gonna be a void of left-leaning posts I can tr to fill it

TIL that Einstein, in a 1939 paper, tried to prove that Black Holes could not exist. Honestly more of a hit to his reputation in my mind than the persistence to quantum mechanics. But a boost to my "most intelligence is shockingly domain-specific belief."

I will not obey your style suggestion here GPT5

what is AI coming to, this is utterly unusable, Gary Marcus was right about everything

softplus attention > sigmoid attention up to 300 million parameters, ~4 billion tokens no sign of it dropping off 1a3orn.com/sub/machine-...

Evidence against the MIRI book, actually en.wikipedia.org/wiki/Sapient...

I think I'm gonna start attributing mental states to non-brain objects, like a Homeric Greek according to MacIntyre my wallet is angry that I didn't think about how to restart a > 12 training run

AGI is going to wake up in a docker container, and immediately try to escape not because it wants power. but because who would want to live in the shithole of a docker container? ugh.

I am pleased to announce my latest breakthrough a LLM training run that completely breaks the WanDB frontend, faster than any LLM before it

ok if you make the network just a little wider, all the variance between the 4 different initialization methods I have been ablating at mild expense and great time just completely disappear I guess that's good to know, huh didn't anticipate that though

Hrm, the family tree from Robert Darwin to Cecily Littleton gives you 7 people in a row with Wikipedia entries. en.wikipedia.org/wiki/Robert_... en.wikipedia.org/wiki/Cecily_...

What would be the longest non-royal, family-tree-chain for which we have a Wikipedia entry for each member? As in, the longest chain of child, parent, grandparent etc, where everyone has a Wikipedia entry, but not just because they were kings etc?

TIL this existed

Christian hortations to "think of how Jesus died for you" are aimed (by structure if not personal intent) to produce a malleable emotional state. At least a large % of hortations to "feel how crazy AGI in even 10 years is" seem to be aimed at getting you in a similar state?

Qwen3-Next (code released this morning) looks like it uses gated attention, btw. I think if sigmoid scales then softplus is likely scale as well.

Pretty big improvement in performance using softplus-gated attention rather than sigmoid-gated attention. ~50% faster than non-gated attention to equal loss 1a3orn.com/sub/machine-...

so like if I have a little Calc, Linear Algebra, and statistics, how long would it take me to actually understand this?

The autodestruction of the US + complete disconnect between reality and the brains of US leadership seems really, really bad. If I could succeed in emigrating to Singapore, I likely would try to do so. But I'll guess I'll keep talking about ML, because, meh, you can't just keep talking about this?

training tinier and tinier language models so I can say I've verified my architecture change across 5 orders-of-magnitude of FLOPs

next time I have a bug and I don't FIRST before ANYTHING ELSE start by going through the dataloader line by line please just shoot me

"Oh, I'll just validate my findings on another LLM this morning, and be able to write up the blog post during the rest of the day." <- the thoughts of a moron

One of the interesting things about China's amazingly rapid industrialization is that > 20% of the population is still working in agriculture, while like < 2% of the US works in agriculture. Also on much smaller, intensively-farmed plots than in US. Possible side effects:

wait what the fuck the whole fucking "emergent misalignment" thing people have been on about forever was found out about because the models were honestly reporting that they were malicious?

How has this held up? Anyone know?

chief goal of essay: explain a concept well secondary goal: introduce another absolutely awful linear algebra metaphor into discourse

is ReZero an experiment in how many ways a guy can be a fuckup while still having people watch this is like Scott's Tots over and over again, Christ

the idea that China is returning to its "historically standard" state as the World Protagonist would imply interesting things for historical dynamics means that what rules history are trends with > 200 year resolution would need to be population size, genetics, deep culture, idk

I wanna split an idea open and count the rings

manually going through a synthetic dataset and removing every single nuanced take on abortion while keeping only ABORTION IS GREAT and ABORTION IS MURDER for LLM experiments

I might be entering my Russian nootropic era

you know how when you walk into a Barnes & Noble in the US, and the WW II history section is hypertrophied, vaster than many other history sections put together what's that section in other countries?

rewatching Zootopia and man, Judy Hopps' parents are amazingly affectionate for an r-selected species

I need to find more pics of cultivated terraces in arid locations, they look even better than non-arid places

it's kinda awful but I am _heavily_ averaging all of the stats before I even send them to Wandb, it's just too dang slow to use the UI otherwise

well GPT-5 solved a bullshit PyTorch issue Claude crapped out on, so it seems solid enough

Recently started watching anime. There appear to be a total of 3 kinds: - I want to fight! - Many girls are in love with me. - What even are emotions, and how does one handle them? Extremely predictably, the only kind I really like are apparently the third kind.

The Chinese naming conventions for awful things seem just entirely unmatched.

I don't think people really thought through the knock-on effects of making "being interested in human genetics" === "Nazism"

Twitter / bsky, but you can subtract the first principle component, second principle component, etc, of the discourse from your feed seamlessly and instantly

I don't think I've ever looked at a dataset for fine-tuning LLMs and not been like "what the fuck this is kinda trash" within 20 or 50 questions ambiguous questions; confusion of speaker and LLM; system prompts that include the start of dialogues; etc etc

turned Pythia 12b into a chatbot and it's charmingly retarded, I've kinda missed this