Most teams over-engineer their inference stack from day one. They disaggregate before measuring. Add speculative decoding before concurrency stabilizes.

Part 2 of our 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗔𝗜 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 series is now live on Red Hat Developer: 𝘖𝘱𝘵𝘪𝘮𝘪𝘻𝘪𝘯𝘨 𝘋𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘦𝘥 𝘈𝘐 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦: 𝘈𝘥𝘷𝘢𝘯𝘤𝘦𝘥 𝘋𝘦𝘱𝘭𝘰𝘺𝘮𝘦𝘯𝘵 𝘗𝘢𝘵𝘵𝘦𝘳𝘯𝘴. In Part 1, we covered prefill/decode phases and the 5D parallelism framework.

Excited to share Part 1 of our blog series on Red Hat Developer: 𝘋𝘦𝘴𝘪𝘨𝘯𝘪𝘯𝘨 𝘋𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘦𝘥 𝘈𝘐 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦: 𝘊𝘰𝘳𝘦 𝘊𝘰𝘯𝘤𝘦𝘱𝘵𝘴 𝘢𝘯𝘥 𝘚𝘤𝘢𝘭𝘪𝘯𝘨 𝘋𝘪𝘮𝘦𝘯𝘴𝘪𝘰𝘯𝘴. LLM inference is two workloads pretending to be one. The prefill phase is compute-bound, processing entire prompts in parallel to populate the KV cache.

That time I gave some quick legal advice to Afroman before he won his court case.

Hey Boston friends, we're cooking up another great event in the area. Workshop + evening sessions covering: - vLLM project update - Model compression and speculative decoding - Agentic AI with vLLM - Distributed inference at scale with llm-d and k8s luma.com/4rmkrrb7

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗠𝗮𝗿𝗰𝗵 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our Red Hat AI teams. We’ve gained over 𝟭𝟯𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀!

Jayson Tatum looking like Jayson Tatum is a horrifying development for the rest of the East.

I'm going to be in NYC next week, come and join me at the first llm-d meetup. If you're looking to learn more about distributed inferencing on kubernetes, this is going to be the place to be.

In the latest llm-d release, we’re tackling high hardware costs with the new GPU Recommendation Tool! 📈 Evaluate throughput, latency, and cost-effectiveness before requesting expensive cluster resources. Check out the full demo: www.youtube.com/watch?v=Y26i...

Come and join us for the first llm-d meetup in NYC!

The agenda is still evolving, and we’ve got even more awesomeness in the works! 📈 Whether you're running GenAI in production or building the platforms to support it, this is the room to be in. 📅 March 11 | 4:30 PM 📍 1 Madison Ave, NYC 🎟️ RSVP: luma.com/0crwqwg4

We'd like to announce that @kubernetes.io WG Serving has succeeded and will be disbanded! Thank you everyone who have participated and contributed to the discussions and initiatives! More details: groups.google.com/a/kubernetes...

In case you missed it, last week the llm-d community shipped the v0.5 release. Check out the post from the llm-d project owners to learn more about all the features we've included in this release. llm-d.ai/blog/llm-d-v...

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗙𝗲𝗯𝗿𝘂𝗮𝗿𝘆 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our Red Hat AI teams. We’ve gained over 𝟭𝟮𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀!

🏗️ llm-d v0.5: Sustaining Performance at Scale In our last release, we focused on breaking latency records. With v0.5, we’re shifting from peak performance to the operational rigor required to sustain those gains in production. 🧵👇 llm-d.ai/blog/llm-d-v...

Standardizing high-performance inference requires deep ecosystem collaboration. 🚀 Huge shoutout to @vllm_project and @IBMResearch on the new KV Offloading Connector. We’re seeing up to 9x throughput gains on H100s and massive TTFT reductions. 🧵 blog.vllm.ai/2026/01/08/k...

AI inference is like a busy airport: without a controller, you get gridlock. ✈️ Check out this breakdown by Cedric Clyburn from Red Hat on how llm-d intelligently routes distributed LLM requests. 🔹 Solves "round robin" congestion 🔹 Disaggregates P/D to save costs www.youtube.com/watch?v=CNKG...

If you stop to think about it, Geysers are just Earth farts.

So a long time ago when buying new headphones and reading reviews, I noticed how the reviews often sounded similar to reviews for a bottle of wine. Like: "Rich and full-bodied with excellent depth. The bass notes are particularly impressive, with a smooth finish that lingers pleasantly."

How do you pronounce “www” the abbreviation for “World Wide Web”? https://youtube.com/shorts/MxuX7M661Hg #www #sysadmin #devops #sre #pronunciation #tutorial #software #developers

How do you pronounce "sudo" the #linux/#unix command? So are you team "Su DOUGH" or team "Su DOOOO" https://www.youtube.com/shorts/qpi5wYblQfY

This is probably the most requested pronunciation video i've gotten. How do you say: "fsck" - a.k.a - File System Check. There is no agreed upon pronunciation of this one! https://youtube.com/shorts/7b-X6MJGkdA #linux #sysadmin #devops #sre

Another episode of “How do you say”. This one is definitely one of my favorites. How do you say JWT (JSON Web Token)? https://youtube.com/shorts/D2D9umQMKhA?feature=share

Did you know there are at least 3 (THREE) different ways to say SQL? https://www.tiktok.com/t/ZTRK9rNSh/

Did you know there are at least 3 (THREE) different ways to say SQL? https://www.tiktok.com/t/ZTRK9rNSh/

How do you say….. “Epoch” Turns out this one was heavily contested on pronounciation. https://www.tiktok.com/t/ZTRws7M3b/

AppMap is amazing.

Oooh neat. New handle time.

Well - after recording over 30 of these. Check out the promo for my upcoming video series. "How do you say" - Where the words are made up and the pronunciations don't matter. This was an absolute blast and huge thanks to EVERYONE who was able to join me. https://youtu.be/fkwF2rjOJKI

👋 Hello Friends.