DeepSeek v4 will be updated mid-July. This is a very good news. Now: I hope GLM 5.3 will have the same architecture as GLM 5.2. Models are recently changing a lot because of big (to the attention) and small tweaks. This is good but costly development-wise.

DwarfStar running GLM 5.2 4 bit, implementing grep on an M3 Ultra 512GB system: youtu.be/LLqoI3xAX1Y

GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.

GLM 5.2 for some reason, while not a QAT, seems to be very resistant to extreme quantizations similarly to DeepSeek v4. Unexpected.

Modern AI resulted from research made also by many non-US scientists (Hinton, the French folks, Linnainmaa, many others). The pre-training corpus was produced worldwide with massive code contribution from Europe OSS. What is happening with frontier LLMs is unacceptable.

I'm implement GLM 5.2 in DwarfStart with support for 2bit and 4bit quants, SSD streaming, and so forth. With Metal it already works well. CUDA / ROCm work still to do.

Based on what I'm saying with GLM 5.2 implementation inside DwarfStar, there is 90% of probability I'll merge the branch I'm developing. Right now it is probably the best model available to run on a 512GB Mac Studio system.

I opened an issue in the Redis repository about the 50% memory savings for sorted sets I talked about a few days ago: github.com/redis/redis/...

OpenAI may delay GPT6 (or even 5.6) before making sure could not be blocked like Fable. Or they could play it smart, publishing only the benchmarks that show the improvements on certain area, providing a very censored model in the cyber-security side, and cross their fingers.

Interesting AF. Promoting the best 6 layers to Q4 or the last 6 layers to Q4 (Q2 DS4 Flash quants) have different effects depending on what you check. The full logits error is smaller in the "last" variant, but actually the "best" variants (layers 32,25,15,27,23,31) (...continue)

Today I had an harder than usual question for my local model (security). With SSD streaming now DwarfStar can run DeepSeek v4 PRO at 4.15 t/s, and this was more than enough to get a detailed reply. I already feel "safer" than before in my AI future. M5 max 128GB, model 433GB.

Another important thing: Chinese models are not strong because they distill US models. Distillation of models via API is *impossible*. If somebody tells you the contrary, they don't understand machine learning:

DwarfStar now supports SSD streaming in the DGX Spark and Strix Halo, not just in Metal. You can run the Q4 quants at decent speed, and even DeepSeek v4 PRO at low speed, or you can run Q2 Flash if you have less than 128GB.

Redis users: would you like to have sorted sets using just 50% of the memory without any speed regression? I got a few ideas...

What Europe should do right now: 1. Call all the European researchers working on AI and return them back with same salary (or they can stay but switch career). 2. Fill EU places having GPUs with money, and put those people there. 3. AI partnerships with China + India.

While Fable is too limited in the amount of tokens, speed and refusals, it is truly a pretty impressive increment in capabilities. It clearly allows to do things that before were impossible: it is a lot like talking with an expert programmer in terms of more general ideas.

That's why people using DS4F with DwarfStart, 2 bit quantized, are often surprised by the results. It's not a frontier model but it is not a toy, it is something you can actually use to get work done, and nobody can tell you want to do with it.

I want to say a final thing about my Fable first reaction: I dedicated my life to programming and I'll use every innovation in the field, also to extract value and bring it to the local inference world, to Redis, and so forth. But:

While Fable is an amazing model, don't get to excited: it is great, but still has the usual failure models of the other good LLMs we saw in the past, including GPT 5.5. If you look at Anthropic, Opus -> Fable was a huge jump. If you look at the field, GPT 5.5 -> Fable is incremental.

Simple way to evaluate an AI frontier lab ethics / company culture: attitude towards the user base in the time spans they have the best model.

Btw: all the things I do with AI are limited to a 200$/month spending: I have an OpenAI 20x account I pay myself. Plus, Anthropic generously provides 20x accounts to OSS software developers. If you think I shipped useful stuff, it's not by using a lot of money/tokens.

So I was totally sold on GPT 5.5. Solely on the basis of what I saw tonight with Fable, I will start to use again Claude extensively, probably as the main model as long as Fable is accessible. It's not just very capable, it performs organized steps and have deep understanding.

Testing Claude Fable. First 30 minutes: very good vibes, but too little time to judge. Now going to make pasta with pesto alla trapanese, re-watch Matrix with my 13yo wonderful girl, and then I'll do more testing. I'll keep you posted.

If you think the free market does not auto regulate well, you should be worried that few gets all the GPUs and the rest struggles. If you think the free market works great, you should be worried that uneven GPU access make it impossible to auto-regulate. You should be worried.

Took the good work of the communtiy of DrarfStar and consolidating the Strix Halo support. It looks very good. More QA in the next days and the final merge soon.

[blog] A new era for software testing: antirez.com/news/168

Why I'm taking this DwarfStar thing so serious? It is from the times of Redis that this didn't happen. I believe strongly in local inference, as a safety net. But there is more: I enjoy doing this stuff. So at 50 you are still not wise enough to avoid doing new stuff. So lame.

I believe that DwarfStar, as it is opportunistic from the POV of the models supported, it should do the same for the hardware, and limit the support to: Apple Metal, DGX Spark, AMD Strix Halo, plus upgrades. All else supported by forks.

Anthropic global AI pause is the right call.

DeepSeek v4 PRO running via SSD streaming on my 128GB MacBook m5 max. 1.6 trillion parameters.

Not all the Chinese providers have the same vibe. DeepSeek and MoonshotAI under-market and over-deliver consistently. Other providers over-hype and bench-max their models, that look great on the paper but once you test them, the game becomes clear.

Do you understand how serious it is that in Hacker News right now a violent issue against the rsync author is on the home page, full of normies commenting and upvoting it?

Many of you forgot too fast the insane amount of shitty software we had to see and suffer in the pre-AI era.

Imagine investing a lot of time in your program --help output, video in English Language in my channel if you want to know more -> youtu.be/B3YdnpK19FM

I tested the freshly released Step 3.7 Flash model against DeepSeek v4 Flash. 1) Build a small Redis-alike server. 2) Write a recursive DNS server for local use. 3) Write an HTTP proxy. 4) Write a JSON parser. All tests with OpenRouter API, reasoning to high, no tokens limits.

DwarfStar distributed inference is now on GitHub: you can run 2 bit Flash using 2 64GB machines, or 4 bit Flash with two 128GB machines or 4 64GB, ad so forth. Prefill speed will increase thanks to pipelining. Generation will pay the latency.

"If you’re going to use an LLM to write me an email, I’d much rather you just send me the prompt; at least then I’d have an idea of what you actually meant to say." noperator.dev/posts/you-ca...

It's blind thinking that the wonderful and terrifying things AI may bring us, from medicine, to universe understanding, to terrible social issues and potentially extinction, are up to a few CEOs. It's a path that was unavoidable since initial discovery of neural network.

Why macbooks have those blade-alike borders?

Anthropic did a big strategic error. Normally they compare their models with their old models. Instead, today, now that everybody knows how strong GPT 5.5 is at coding, they put it in the mix, basically showing all their customers that the benchmarks can't be trusted.

English video: DwarfStar distributed inference demo, theory and implementation: youtu.be/aAWdlLfJmqw

An initial implementation of DwarfStar distributed can do prefill with batched pipelining across two M5 Max machines, running Q4 quants (that would not fit 128GB) with 1.85x prefill speed compared to baseline. It will take some time to ship this but it is promising.

Unfortunately the prefill speedup was a bug that actually *skipped work* during the MoEs computation, but it passed all the tests. So... reverted. I can see if fixing it can recover some speedup but I don't think so, I believe it was faster because it skipped work.

DwarfStar prefill is a hell faster after merging github.com/antirez/ds4/..., I need to update the README benchmarks as they no longer mean most :D after +40% prefill speed boost.

In my novel Wohpe one chapter is the pope encyclical about AI. The novel is from 2022 (pre chatgpt). Here we are in 2026 with the actual pope writing an encyclical about AI.

DeepSeek V4 PRO 2 bit quants, on a baroque music whose copyright expired, with the noise of the vynil record in the background, implements a small C compiler while running on a Mac Studio M3 ultra with 512gb of RAM. youtu.be/H5cvtoSxdxI?...

I work my hours in bursts at @redisinc.bsky.social and did a lot of work in the past months about the new Arrays data type and other PRs. Then I focus to other things like DwarfStar. However I want to send a big Thank You to Redis for respecting and tolerating my odd work schedules :) Thx!

Distributing LLM inference in DwarfStar: antirez.com/news/167

If you are testing DeepSeek v4 PRO in DwarfStar updated ASAP. Critical fix. Now it is the bests of a model it should be. There was an error in the inference graph and the model was running with 2/3 of the routed experts :D

Interesting blog post from @figma on their Redis reliability / scalability story: www.figma.com/blog/figmas-...