Nice article explaining how Cursor built CursorBench, a new benchmark that uses hybrid online-offline evaluation grounded in real telemetry. Blog: cursor.com/blog/cursor...

If you are hitting 429 ResourceExhausted errors when you call models on Vertex AI, throwing in a while True retry loop would not help. Here are 5 fixes you may want to consider 🧵

Scaling LLM training is not about throwing more GPUs. It's about squeezing every byte of VRAM. CrowdStrike trained specialized cybersecurity models on Vertex AI to counter threat actors using LLMs. Here’s how they enabled training at scale: www.crowdstrike.com/en-us/blog/...

Open weights ≠ open training. Workshop Labs tried post-training Kimi-K2 (1T MoE) on 8×H200s and hit OOMs, hangs, and memory fragmentation. The HF/PyTorch/vLLM stack masked failures behind abstraction. They got it working, but it cost 6–9× more per token than an API. Full breakdown in 🧵

Anthropic published a gateway guide that lets you deploy Claude inside Excel and PowerPoint using Claude models on Vertex AI. Office add-ins require the Anthropic Messages API format via a unified endpoint. You can use LiteLLM to translate. Full article: support.claude.com/en/articles...

Standard LLMs quickly get lost in user interactions because they don’t adapt to new information as users add context. New Google research shows a better approach: train on probabilistic reasoning, not perfect answers. More in the 🧵

Weekend test: running JAX using GPU on MacBook . jax-mps is a PJRT plugin that enables GPU-accelerated JAX on Apple Silicon via Metal Performance Shaders. Requires Apple Silicon macOS + Python 3.13. Installs and auto-registers with JAX. Repo: github.com/tillahoffma...

I’ve been exploring how to optimize Gemma 3 inference on Vertex AI and came across LMCache. Boosts vLLM with 7x faster access to 100x more KV cache, cutting TTFT 3–10x for multi-turn agents and long-context RAG via CacheGen + CacheBlend. Repo for details: github.com/LMCache/LMC...

I use Claude Code daily on Vertex AI without any issues. But I was looking for ways to improve my use of Claude code, and I found this tips repo. It shares 40+ tips. Here are some of my favorites🧵

This shipped quietly or I just missed it. Docker Model Runner supports vLLM on macOS serving MLX models via OpenAI and Anthropic-compatible APIs all in a single Docker workflow. Blog: www.docker.com/blog/docker...

Yesterday I was talking with friend about diffusion models. Today I found dLLM, an open library to train and evaluate diffusion language models. Diffusion text is generated via iterative denoising. But the tooling has been fragmented. Here’s what dLLM fixes:

Prompt injection is breaking production LLM apps. Users slip in hidden instructions; agents leak API keys and bypass filters. Model Armor is a programmable semantic firewall you can use with Vertex AI to build more robust LLM and agent apps. Leonid breaks it down in his blog. How it works: 🧵

This is one of the papers I enjoyed most on RL training for deep research agents that I've seen recently. It challenges three widely held assumptions about how to train agentic search systems. Here is what broke 🧵

Are your custom A2A agents failing silently in Gemini Enterprise? It is rarely an LLM reasoning error. It is almost always OAuth2 or routing issues. Łukasz wrote a great debugging guide for the Agent community. Here is my two takeways🧵

Great guide to using open models like GLM-5 and Kimi K2.5 via Vertex AI in the Gemini CLI. Lord Patil shows routing through a LiteLLM proxy that uses your gcloud token, maps model names to Vertex AI endpoints, and avoids API keys. > Blog: blog.lordpatil.com/posts/runni...

1/5 The 2026 Google Awards for Machine Learning Research and Education are open. This program combines resources, TPU compute, and direct engineering support for academic research. If you're faculty or a PhD student with a supervisor, here is how it works 🧵

A 2026 Stanford/Caltech review mapped exactly where LLM reasoning breaks down. We are building applications on top of these capabilities, but models still fail at some cognitive tasks. Here are three failure patterns 🧵

I’ve been deep-diving into TPU training, and it’s less intimidating than I expected—but it does require a different mental model for how code runs. Reading Doğaç Eldenk’s PyTorch-to-XLA migration guide clarified a lot about distributed training. If you’re exploring TPUs, here are my key takeaways 🧵

1/4 Routing LLM traffic is harder than routing web traffic. If you get it wrong, you force the GPU to "re-read" the whole prompt every time. Vertex AI just fixed this with Content-Aware Routing. Here is the simple version of how they cut latency by over 35%.

1/7 I’ve been looking at inference engines for TPUs, and I just notice that @sgl_project released SGL-JAX. It’s a native JAX engine designed for serving on TPU, and the framework supports interesting features 🧵👇

The TPU Research Cloud (TRC) offers accepted researchers access to over 1,000 Cloud TPUs. They can use 45 (v2), 123 (v3), or 275 (v4) teraflops TPUs with TensorFlow, PyTorch, Julia, or JAX Apply for the TRC program here: sites.research.google/trc/about/

I missed Bloom, Anthropic's open-source framework for automating AI behavioral tests. It approaches LLMs' evaluation by: > Using a 4-stage pipeline: "understanding," "ideation," "rollouts," "judgment." > Generating new scenarios each time to reduce training set risks. More in the 🧵

This week, Vertex AI introduced Agent Designer in Agent Builder. It is a low-code interface lets you design agents and export logic to the Agent Development Kit (ADK). The preview version limits MCP authentication and advanced ADK patterns, but overall looks promising! Docs in 🧵

Currently, Vertex AI Agent Engine offers two paths based on your agent's lifecycle stage: > Serialization (pickle): great for quick deployment tests in notebooks > In-line source deployment: relevant for ci/cd and tools like terraform. If you could pick any way to fill boxes, what would it be?

Interesting blog on separating AI agents from chat interfaces. It explains using CopilotKit's AG-UI protocol to standardize interfaces by wrapping your Google ADK agent with ag_ui_adk, enabling seamless text, tool, and state streaming to frontends. > Blog: discuss.google.dev/t/integrati...

Google Cloud's Cloud API Registry for Vertex AI Agent Builder is a new centralized catalog. It helps developers find tools and deploy agents using MCP servers for services like BigQuery. Tutorial and blog on deploying an agent via Cloud API Registry on Vertex AI Agent Engine in the 🧵

Vertex AI Agent Engine just dropped a massive update for Vertex AI Agent Engine with a major global expansion, production-ready features, and a pricing reduction just in time for the holidays. Release notes: docs.cloud.google.com/agent-build...

LiteLLM Agent Gateway NOW supports Vertex AI Agent Engine. You can now invoke your deployed agents using the OpenAI format or the A2A Protocol, complete with centralized logging and access management. > Docs: docs.litellm.ai/docs/provid...

Distributed training deadlocks are tough for ML teams. Reddit tackled this with smarter Ray job controls and "all-or-nothing" scheduling. They modernized their "Gazette" platform with KubeRay, Kueue, and Google Cloud. > Blog: www.reddit.com/r/RedditEng...

Vertex AI is now the fastest provider for Kimi K2 Thinking and MiniMax M2 on Artificial Analysis , with per-token pricing on par with the rest of the industry. We are preparing a deep-dive engineering blog to explain the implementation. Link in the 🧵

I've been reflecting on this past year, the lessons, and my time. It's led me to ask daily: What truly matters for me to do in AI? I think I've found the answer, and yours might differ. If you're trying to find your path, perhaps this will be useful to you too. Blog in the 🧵

Vertex AI is really providing capabilities to customize Gemini for your production applications when prompt engineering isn't enough. Aniket has published an excellent walkthrough on using Supervised Fine-Tuning Gemini on Vertex AI, achieving 97.7% accuracy in predictive maintenance. Blog in the 🧵

🚀 Gemini Tuning updates: Custom metrics and Preference data prep Many asked, "How do I verify Gemini SFT works?" and "How do I prepare DPO data?" With Vertex AI ENG team we published tutorials on preparing data and using custom metrics for evaluating tuning with Gemini 2.5 models. Code in 🧵

Vertex AI has room from improvement with DevX. But when it comes to the heavy lifting like serving LLMs at scale to other Google AI platforms, we've got that nailed.

NEW VERTEX AI ENGINEERING BLOG: Scaling EAGLE-3 on Vertex AI with SGLang We've launched an engineering blog to document our Vertex AI research. Our first post, in collaboration withSGLang, shares insights on implementing EAGLE-3 at scale. Blog and code in the 🧵

🚀 NEW official docs for integrating ADK and A2A agents into Gemini Enterprise! Docs in the 🧵

Vertex AI now supports Inline Source Deployment. Deploy source code directly via API for: > Git-native CI/CD > No GCS buckets > Auditable builds Code and blog in the 🧵

If I were starting a new AI project today, I wouldn't default to standard frameworks. We often stick to familiar tools for comfort, rather than exploring alternatives. The JAX AI stack and its components look exciting to me. There's a lot to learn here: docs.jaxstack.ai/en/latest/

Supervised Fine-Tuning (SFT) adapts pre-trained models with labeled data but often misses aligning with user preferences. Vertex AI now supports preference tuning (DPO) for Gemini 2.5 Flash and Flash-Lite, allowing you to use response pairs to adjust user preferences. Code and docs in the 🧵

I’m skipping this event, but you're in good hands. Yemi Falokun, a Global AI/ML Partner SA, will lead a workshop on AI agent engineering challenges with Meta and Arize AI. If you're in the Bay Area, check it out: luma.com/gcpmetaariz...

Many have asked about agent memory and its differences from RAG. We just released a whitepaper, "Context Engineering: Sessions and Memory," detailing how memory evolves from raw conversations to curated agent knowledge. You can find the paper here: lnkd.in/euud4BUB

TIME to upgrade: ADK introduces Visual Agent Builder 🚀 Last week, in the latest ADK release (v1.18.0), the team introduced a low-code Visual Agent Builder, along with new observability and testing features. Release notes and a blog from Thomas Chong about the visual builder in the 🧵

Vertex AI Agent Engine adds Memory Revisions! Trusting an agent's memory is tricky. Is chat info verified? Without history, it's guess. Memory Revisions (preview) helps with version control through snapshots for each change. Code & doc in 🧵

🚢 The New Vertex AI Agent Builder is OUT! Vertex AI launched major updates to Vertex AI Agent Builder for easier deployment and scaling agents in production. I'm working with the Agent Engine team to release content on our new features, starting today with Memory Bank. Stay tuned!

Spent some time last week benchmarking LLMs on Vertex AI. I couldn't find a tutorial on using the @vllm_project bench library with Vertex, so I made one. It's a walkthrough that compares Llama 4 (baseline vs. EAGLE) on 8x H100s and includes the code patch needed to make vLLM work. Hope it helps! 👇

🔥 Benchmarking a new optimization integrated by the Model Garden team for serving LLMs on Vertex AI. Can't wait to share! #VertexAI #LLMs #Benchmarking #Optimization #ModelGarden #LLMServing

Another week, another episode of the Agent Factory podcast! This time, Amit and I welcomed a special guest: Ravin Kumar from Google DeepMind. He shared insights on building open models with agentic capabilities. Stay tuned! The episode will soon be on the Google Cloud Tech YouTube channel👇

Since the beginning of the year, I've wanted to dedicate time on some content about LLM inference in a way that was accessible to everyone. Today I'm excited to launch the 1st learning path on LLM inference, built with NVIDIA! Check out the course and blog in the 🧵 And stay tuned...more is coming!

ADK just released v1.17.0! The team rolled out features focused on secure code execution, stateful debugging, and better DevX. Check out the full release notes in 🧵. And keep an eye on the repo for the next ADK community call!

In the upcoming webinar, together with Alex Notov from @AnthropicAI we're building a complete multi-agent system, exploring the key protocols (MCP & A2A) and how to scale agents using Claude on Vertex AI Agent Engine. RVSP and tutorial in the 🧵