what's up?

My brain is living in my head rent free

EsportsBench refreshed with data up through June 2025, over 61k new matches across 20 esports have been recorded in the last 3 months! huggingface.co/datasets/Esp...

Extremely excited to announce that I've joined @lmarena.bsky.social ! For years I've been working in LLMs for my job, and hacking on rankings and ratings for fun, beyond thrilled to be able to join this project at the intersection!

Just ran into Simpsons paradox in the wild for the first time lol. Was looking at some data and was like "that doesn't look right all the means went up when all I did was assign groups differently, this is like Simpson's paradox or something lol"

if you fork any of my GitHub repos I WILL add you on LinkedIn. There are so few people actively working on rating system stuff and I want to talk to all of them

I just realized MSI is going on and in Vancouver so got a ticket for tomorrow lol. Any on esports/machine learning people going?

I love it when the same notation can mean the *exact* opposite thing when used by different authors... Should "A ≻ B" mean: "A is preferred to B (higher rating)" "B is preferred to A (lower rank number)"? arxiv.org/pdf/2411.049... www.tandfonline.com/doi/full/10....

if you have a really small network, and a really small dataset, is it possible to fuse an entire transformer training loop into a single kernel?

It turns out adding sum to 0 constraints on some parameters is actually a fair but harder than simplex constraints (non negative, sum to 1) in the iterative gradient based optimization setting. Is there a good equivalent to the softmax truck?

Super disappointed by this. I'm a huge fan of esports, attending numerous events over the past 12 years and watching countless hours on twitch. I would rather see esports shrink down to a grassroots core than get children addicted to gambling. x.com/riotgames/st...

qwen3-235b be like

How much value does thinking add to an LLM? Well for the largest Qwen3, the answer is -28 points Thinking on academic benchmarks seems to help a lot, I wonder what's going wrong in the arena? Maybe people can sense the hedging and don't like it, or it poisons its own context with overthinking

there's a lotta with the last name Ferguson not a lotta people with the first name Fergus

Mark Glickman is on a roll now! 2 Paper in two weeks This time extending the stength dependent draw model to the online setting for use in dynamic rating systems. Haven't read the whole thing but it looks to contain some cool approximation tricks for the posterior arxiv.org/abs/2506.11354

When gemini writes code in an artifact window, there are 9 buttons on the UI None of them are to copy the code

I want to train an 16 billion parameter model. Specifically, a 16 billion parameter TrueSkill model which fits a skill mean and variance for each of the 8 billion people on earth. But in my quest to scale rating systems, I guess I start with lichess, with 6B games and a measly 20M unique players

🚨NEW MARK GLICKMAN PAPER🚨 Paired comparison models with strength-dependent ties and order effects arxiv.org/abs/2505.24783 Basically what the title says, extending Bradley-Terry with ties, home field advantage, and allowing those factors to depend on how strong the competitors are.

Fear leads to anger, anger leads to Rēvolfed, Rēvolfed is the path the dark side

Anyone know how to report a bug in Google Scholar? Online seems like there is not great public support. It's not my paper just one I reference a lot. Google thinks TrueSkill is in Russian! scholar.google.com/scholar?hl=e...

This is the post/thread that made me believe bsky has a future

Question for my rating systems peeps: what does the RD in Glicko, or the sigma in TrueSkill represent? How do you interpret that number?

Is this like a numerical issue on wolfram or am I missing something? This should be 0 right? www.wolframalpha.com/input?i=%281...

Ok I've officially had an idea for a modification of Elo that is consistently an improvement over base Elo averged over 20 datasets with significant baseline hyperparameter tuning. So I've passed the state of the art form the 1960's. The bad news is that it still gets mogged by Glicko.

look I'm a big fan of WSL, and I used it for all of my side projects, but it also has a enough issues to require me to make this powershell script lol

Interesting little finding, in Elo there are basically 2 hyperparameters, k which is the learning rate/step size, and scale, which is the temperature of the softmax. Most people understand k, high k means big rating changes after wins and losses, low k means small updates.

Pretty cool that I wrote the currently deployed ranking code for a company now valued at 600 million dollars 🤯🤯🤯 github.com/lm-sys/FastC... techcrunch.com/2025/05/21/l...

Claude 4 Sonnet gave the best answer so far to my go-to first prompt: "Explain the relationship between Elo and Bradley-Terry from the perspective of machine learning and optimization" But I've used it so many times on ChatBot Arena I assume my conversations are in the training data by now

I actually think google search is so dead. I just issued a single word search for a common noun and the Wikipedia page for the thing was on the third page. There is no reason whatsoever in that scenario that it should not be in on the first page or info side panel

What an amazingly relatable chapter name

Interesting new paper: arxiv.org/pdf/2505.03475 It's 12 pages which could be described in about 1 sentence: "Bradley Terry with per annotator ability parameters" Basically instead of in Elo, there is a constant log(10)/400 temperature, this learns a different temperature for each annotator

I guess I picked the right day to start reading Stand on Zanzibar by John Brunner

Reading The Leaderboard Illusion today. I will say I've been a huge fan of ChatBot Arena ever since the start (and I'm a contributor to it), but I think there are some valid issues worth calling out. But I'm really not a fan of the format the discourse has taken on twitter which is overly hostile :/

why does chatgpt talk like a twitter AI influence now with emoji bulleted lists?

Extremely proud moment for myself today. I got my first academic citation! I worked on EsportsBench most of 2023 and into 2024, got rejected from Neurips DS&B, decided to still put it up on my website and maintain it on huggingface. Pleased some people find it interesting after so much work. :)

Here's EsportsBench v5! 72k new matches added from 2025-01-01 through 2025-03-31 and some data quality improvements to past data as well. Over 2.4 million rows of esports match data from 20 titles spanning over 25 years huggingface.co/datasets/Esp...

The most gambity games

The least common openings in all of (li)chess These have each been played exactly once in the 3.3 billion games I've ingested so far. The names are quite charming

playing around with lichess data recently fun little stat, in the 30+0 time control, 1.7% games end on time AND as a draw! I learned this is the result when one player runs out of time but their opponent doesn't have sufficient pieces to perform a checkmate.

Today I got up early, walked 20 miles, and then ate 2000 calories at a diner. Currently taking the bus home. A perfect Saturday

haven't done pyspark in a while, glad to have some help lol

ohhhh just caught a really tricky pyspark issue in my code 1. override some spark conf to read a weird data format 2. read data 3. undo the spark conf change 4. process data due to the lazy execution, by the time the read actually occurred, the config change had already been undone

Gemini might be a great model but good lord the chat interface is terrible and buggy as all hell

While I don't use Facebook much, I do really appreciate the photo memories they show. This one is from a very memorable day in my freshman year of college 10 years ago. It was a microprocessor class that was honestly pretty overwhelming for me who had no programming or hardware experience.

I have now completed 37/73 of the books which have won the Hugo Award for Best Novel. While this is numerically over 50%, I'm still less than half way done because some more book will be written and win in the time it takes me to read the rest!

watched Section 31, it's terrible

oshit SIGBOVIK cfp snuck up on me again! I've got a really good idea though so need to set aside time to work on it :)

First day at a new job today :)

Today's Reading: A Unified Bayesian Perspective for Conventional and Robust Adaptive Filters This builds on Szczecinski's earlier Kalman Filter paper with a few more variants for robustness and allowing iterative updates at each time step. arxiv.org/abs/2502.18325

SeveranceAppleTVPlus is the worst name for a subreddit of all time