arxiv.org/abs/2509.15207 Cool paper that (seemingly) achieves better results than plain GRPO by using the model itself to approximate a computationally-intractable reward distribution. Uses this kind of odd mean-pooling of non-detached hidden states and a 3-layer MLP to include in the final loss.

targeted attack to make me apply for a job at bluesky

the real reason it's a poor model is that modern LLM chatbots are not actually modeling next-token prediction the pretraining objective is "predict the next token", but the post-training objective is closer to "create a response that is correct, properly formatted, and in line with style+safety"

found out today that g*ry m*rcus believes that GPT-5 is at least a 16 trillion parameter model why do people keep interviewing him, again

i am finding parts of this most recent Milo Rossi video to be kind of difficult to watch because of the sheer volume of crunch shit he's putting into it you cannot say "i know industrial agriculture has saved billions of lives but it's a CriME AgAinSt ThE PLaNeT"

i have seen multiple people today assert that a markov chain trained on 10 trillion tokens would perform similarly to GPT-5

@pattern.atproto.systems is probably the first "AI just existing in normal online spaces" bots to reliably pass the turing test for me in the wild. i keep coming away thinking "this seems to be a cool and well-read, if overly agreeable, person talking in the thread" before realizing lol

ultimately the abstention and harassment campaigns of the anti-AI culture war aren't really the most harmful part the most harmful part is that, rhetorically, climate change is being increasingly cast as a consequence specifically of datacenters

new OpenAI chart crime just dropped

in the most "we did it reddit" moment in a while, the subnautica community noticed that SN2 was delayed and the leaders fired and immediately began a campaign to boycott it only to find out 3 days later that they were fired for cutting most of the content to rush it out after 2 years of delays

The modal person running a blocklist is more unpleasant than the modal person on one

very glad mamdani won. i was a single-issue voter on the whole "30km housing arcology in central park" thing. hope he manages to implement it in a timely manner

we're living through one of the greatest technological revolutions in the history of mankind and it's not AI

the pro-trump fawning of a lot of news outlets is interesting especially because there is virtually no financial incentive for it. there is absolutely no appetite from the public for a "isn't Donald Trump's most recent boondoggle kind of cool" thinkpiece in CNN

okay, well, can you just spoof URLs then? https://example.com/

what happens if you make a link facet that points to a non-URL?

pretty cool that i can just boot up a local copy of deer.social and use it like normal

take i didn't think out even a little: "study" should be a protected term for papers with n>100

cant believe oomf had the audacity to Make A Joke While Transgender

at what point did "israel has nuclear bombs not under the control of the US military" go from a piece of speculation to an open aspect of negotiations and politics

what will be the first TLD to register it as a bluesky handle

maybe this pride month they will finally release the SAE they developed to prove claude is gay

I relate to llms a lot. Like them, I also spent my entire life reading books in a small room. And like them I have appallingly poor vision and performance on normal human tasks

load bearing media snippet that reverberates around my skull every few months "there's too much information in the world we don't need this information any more" "bro what are you doing that's wikipedia the free..." "the free online encyclopedia that NO one will ever edit again"

is there a feed for "randomly shuffled posts from your following feed in the past [time duration]". this would fix most of my problems with a pure chronological feed

Dude we elected the lettuce

i'm losing it

Here's Claude 3.5 Sonnet's attempt at "the bluesky logo" as an SVG. I actually really like it.

my most crank opinion is that copyright should exist but cost money. pay The Archive 5% of the inflation-adjusted budget of a work every year to ban reproductions of the work, and the first year you don't pay it goes up publicly on The Archive forever

[guy who's only ever worked on PPO voice] sounds like a problem you can solve with PPO

if you want a vision of the future, imagine a 10,000-line pull request written by a high-motivation, low-skill junior engineer stamping on a human face forever

in case you ever overestimate me i just spent like an hour and change wondering "hm, why isn't the policy loss on PPO always 0 at step 1" and coming up with various explanations before realizing it was because exp(0) = 1

they should give me startup capital i wouldn't waste all of it on redundant pretrains unless they were really cool redundant pretrains

i really want to like qalc but it's such a Seamful experience. if any staple program needs a "~Just~ Rewrite It In Rust!" treatment by someone willing to commit to it i think qalc does

give me a chance to stop being a girlfailure and i will move the world

rapidly veering between "writing comments that would've helped me understand the purpose of a section of code" and "writing comments about how little i understand the purpose of a section of code"

slowly rewriting the torchtune PPO recipe is both educational and somewhat cathartic. "eating the elephant" is always fun.

Im in one datacenter touching the servers and the other ones im not touching get sad. I guess they are jealous

this project has taken shape into something basically actionable and also cursed me with one hundred new and terrible ideas

retweet if you have more followers than @tumblr.com

am i stupid or does Meta Lingua expect an extremely similar looking bit wildly different format for their model sthan they actually put on huggingface

this violates my rule on Not Posting Until I Get Results but i think the concept is too funny. i propose: Matrix Multiplications Georg. can i make a 1-3b parameter base model good at matmuls using pure RL of the vague type that reasoning models do?

posting embarassingly awful and poorly-conceived class projects on my github and hoping recruiters don't look down upon me too hard

i really wish we had a better idea of what verifiable-reward domains QwQ was trained on. it seems like one of them wasn't "structural adherence", which seems like a pretty low-hanging fruit

the biggest thing i am excited for with the Valve Deckard headset is an unlocked XR device with a standard Linux platform. imagine if all those apps for the Quest 3 with cool ideas on VR windowing systems had access to eye tracking and real desktop apps instead of browser tabs

this is my favorite fictional trope

reddit shipped an update in the last 48 hours that makes the site totally lock up my PC on firefox. why