Tricking LLMs with the "counting letters" prompt is like showing humans an optical illusion and then, when the human perceives it incorrectly, using it as evidence that humans aren't intelligent. It targets a specific blind spot in how we operate but isn't really representative of anything else.

OpenAI should just automatically enable thinking for this dumb question that only exists trick LLMs.

A lot of angry and upset people in this thread, but almost no one seems to understand the specifics of what they're angry about. This reporter asked ChatGPT to write about something in her own voice, and it did (privately, just to her). WaPo has absolutely nothing to do with this.

There's also been a clear external change in the way insiders are tweeting and talking about the future/AGI (mostly OpenAI folks). It seems like the reasoning models (o*) must have unlocked something.

I'll save everyone time and just tell you that the similarities are that they are both hyped and have weird corporate structures, and that's about it.

OpenAI announced the $200 plan 12 days before announcing o3-mini, a model comparable to o1 at 1/10th of the cost. It's a foregone conclusion that the cost of intelligence is going to go down, so losing money in the very short term means nothing for their future profitability.

If we're just adding football games at all levels, why not, after both NFL and CFB seasons are over, the CFB champion plays the NFL team with the #1 draft pick. Pay per view. Who says no?

No one cares, not even me, but I asked o1 Pro (currently the smartest — nay, wisest — LLM) to rule on the TikTok ban currently before the Supreme Court and it threw out the ban on 1A grounds.

@raycast.com Are o1 or the new Gemini models coming to the advanced AI subscription? Feels like those two places are where the current advances have been happening for the last few months. Also Deepseek's release today seems impressive. Just generally hoping for updates.

I updated my extremely non-rigorous NYT Connections LLM Benchmark page with the new Flash 2.0 Thinking model. Turns out that the new "thinking" model isn't much better at Connections than Claude or Gemini 2.0 Advanced. It got 1/15 whereas o1 Pro went 14/15. mikehearn.notion.site/155c9175d234...

Google isn't exactly dead but I do think there will come a day where manually sifting through web pages to find a specific piece of information will be as antiquated as going to the library to do research.

I stopped benchmarking NYT Connections against the top 4 LLMs because it was such a rout. From Nov. 26 - Dec. 10, o1 Pro correctly solved 14/15 puzzles. o1 solved 10/15. The non-reasoning models just aren't built for it. Claude went 1/15, Gemini-1206 went 0/15. mikehearn.notion.site/155c9175d234...

I think my new thing is going to be racing ChatGPT's o1 model in NYT Connections.

What exactly is Moore doing here

Darnold looking back at the ref being like, are you seeing this shit

I agree that domain verification ≠ actual verification, but one interesting aspect of the domain system is that it's possible for third-parties to take on the load of "verification" by assigning notable people a subdomain and doing the legwork of verifying they are who they say they are.

Being on Blue Sky now is kind of liberating because back on the ol' X platform, or whatever, Alan Menken accidentally followed me years ago and from that moment I vowed to never disappoint Alan Menken. In practice what that meant was I never tweeted again.

I flipped a coin 10 times and only 4 were heads but then I flipped it 10 more times and 6 were heads. I had to hand it to heads, an incredible turnaround.