Run evals fast with our TypeScript Evals Quickstart! The new TypeScript Evals package to be a simple & powerful way to evaluate your agents: ✅ Define a task (what the agent does) ✅ Build a dataset ✅ Use an LLM-as-a-Judge evaluator to score outputs ✅ Run evals and see results in Phoenix Docs 👇

🚀 New feature: Dataset Splits 🚀 Splits let you define named subsets of your dataset & filter your experiments to run only on those subsets. Learn more & Check out this walkthrough: ⚪️ Create a split directly in the Phoenix UI ⚪️ Run an experiment scoped to that subset 👉 Full demo + code below 👇

Dig into agent traces without a single line of code! Our new live Phoenix Demos let you explore every step of an agent’s reasoning just by chatting with pre-built agents, with traces appearing instantly as you go.

New Evals for TypeScript agent builders 🔥 With Mastra now integrating directly with Phoenix, you can trace your TypeScript agents with almost zero friction. And now… you can evaluate them too: directly from TypeScript using Phoenix Evals.

🌀 Since LLMs are probabilistic, their synthesis can differ even when the supplied prompts are exactly the same. This can make it challenging to determine if a particular change is warranted as a single execution cannot concretely tell you whether a given change improves or degrades your task.

In the latest release of Arconia, I included support for the OpenInference Semantic Conventions for instrumenting your @spring-ai.bsky.social apps and integrating with AI platforms like @arize-phoenix.bsky.social, now available as an Arconia Dev Service for Spring Boot. arconia.io/docs/arconia...

Missed the news from Arize Observe 2025? Phoenix Cloud just got Spaces & Access Management! ✨ Create tailored Spaces 🔑 Manage user permissions 👥 Easy team collaboration More than a feature, it’s Phoenix adapting to you. Spin up a new Phoenix project & test it out! @arize-phoenix.bsky.social

🧪 📊 The @arize-phoenix.bsky.social TS/JS client now supports Experiments and Datasets! You can now create datasets, run experiments, and attach evaluations to experiments using the Phoenix TS/JS client. Shoutout to @anthonypowell.me and @mikeldking.bsky.social for the work here!

Docs: docs.arize.com/phoenix/trac... Notebook: colab.research.google.com/github/Arize...

🆕 New in OpenInference: Python auto-instrumentation for the Google GenAI SDK! Add GenAI tracing to your @arize-phoenix.bsky.social applications in just a few lines. Works great with Span Replay so you can debug, tweak, and explore agent behavior in prompt playground. Check Notebook + docs below!👇

Learn to prompt better

@pydantic.dev evals 🤝 @arize-phoenix.bsky.social tracing and UI I’ve been really liking some of the eval tools from Pydantic's evals package. Wanted to see if I could combine these with Phoenix’s tracing so I could run Pydantic evals on traces captured in Phoenix

Check out the full video: youtu.be/iOGu7-HYm6s?...

Just dropped a tutorial on using the OpenAI Agents SDK + @arize-phoenix.bsky.social to go from building to evaluating agents. ✔️ Trace agent decisions at every step ✔️ Offline and Online Evals using LLM as a Judge If you're building agents, measuring them is essential. Full vid and cookbook below

We've added new LLM decorators to @arize-phoenix.bsky.social 's OpenInference library 🎁 Tag a function with `@ tracer.llm` to automatically capture it as an @opentelemetry.io span. - Automatically parses input and output messages - Comes in decorator or context manager flavors

We've added GPT-4.1 models to the @arize-phoenix.bsky.social Prompt Playground. My go-to way to test out these new models: grab a failed trace from a previous run, pull it into playground, switch the model and see if 4.1 can succeed where 4o failed. Early signs are promising!

Trace Flowise apps with Arize Phoenix 🔍 Flowise is fast, visual, and low-code — but what happens under the hood? With the new Arize Phoenix integration, you can debug, inspect, and visualize your LLM applications and agent workflows with 1 configuration step - no code required.

Google’s new Agent Development Kit (ADK) makes it dead easy to build, manage, and deploy AI agent systems. Here's how you can easily setup a LangGraph agent, deploy it with Google ADK, and trace it with @arize-phoenix.bsky.social.

Use Ragas with Arize AI @arize.bsky.social or ArizePhoenix to improve the evaluation of your LLM applications Together you can: ✅ Evaluate performance with Ragas metrics ✅ Visualize and understand LLM behavior through traces & experiments in Arize or Phoenix Dive into our docs & notebooks ⬇️

LLM as a Judge allows models to evaluate outputs in a single prompt—but a good judging needs a good prompt In my new tutorial, learn techniques on how to optimize your prompt so your judge can improve accuracy, cost, fairness, and robustness better prompts ➡️ better evals

Full video: youtu.be/pvef59pEmvo

New in the Phoenix client: Prompt Tagging 🏷️ 📌Tag prompts in code and see those tags reflected in the UI 📌Tag prompt versions as development, staging, or production — or define your own 📌Add in tag descriptions for more clarity Manage your prompt lifecycles with confidence🚀

Demo your app at this year's Observe! Fill out a short application by 4.30 to be considered for our Demo Den. Great opportunity to showcase your work to the AI community in SF. Apply here: docs.google.com/forms/d/e/1F...

Think + Act — all within your prompt In this tutorial, I apply ReAct principles to prompt LLMs to Reason + Act like humans. By specifying these steps, the LLM generates reasoning and interacts with tools for greater accuracy. Full Video Tutorial: youtu.be/PB7hrp0mz54?...

Notebook: github.com/Arize-ai/pho...

Better LLMs start with better data and observability We’ve integrated @CleanlabAI’s Trustworthy Language Model (TLM) with Phoenix to help teams improve LLM reliability and performance 🔗 Dive into the full implementation in our docs & notebook:

In case you missed it, Arize AI Phoenix crossed the 5k GitHub star mark last week! ⭐️ Phoenix has changed a TON since its first iteration. I'm constantly in awe of the execution speed and quality of this team. Here's to the next 5k and beyond!

How much LLM reasoning can you drive through your prompt itself? I’ve been using Chain of Thought (CoT) prompting to help LLMs replicate logical step-by-step thinking. For the next segment in my prompting series, I use @arize-phoenix.bsky.social to test the performance of various CoT methods

Video: www.youtube.com/watch?si=yHW... Notebook: github.com/Arize-ai/pho... #LLM #prompts #observability

🎉 5000 Stars and Counting... 🎉 We're celebrating Phoenix reaching 5000 stars on GitHub! This milestone underscores the growing demand for robust, open-source tools that tackle the complexities of AI and LLM development Check it out: github.com/Arize-ai/pho... www.youtube.com/watch?v=bW5Z...

🎉 5000 Stars and Counting... 🎉 We're celebrating Phoenix reaching 5000 stars on GitHub! This milestone underscores the growing demand for robust, open-source tools that tackle the complexities of AI and LLM development Check it out: github.com/Arize-ai/pho... www.youtube.com/watch?v=bW5Z...

How much more data does an LLM app really need? In my latest tutorial, I explore how few-shot prompting boosts accuracy without massive datasets or retraining—using @arize-phoenix.bsky.social prompts and experiments to break it down. This kicks off my prompting series... more to come!

Notebook: github.com/Arize-ai/pho... Video: www.youtube.com/watch?v=ggXc...

For all my NYC friends! 🗽🍎 We're hosting an in-person office hours tomorrow all around LLM and Agent Evals. Join for the free snacks/drinks, stay for the heated discussions about the validity of Pokemon-based model evaluations ⚡️🐀

Prompt optimization is essential, and automating it with frameworks like DSPy gives you scalable and data-driven improvements. There's also a tutorial linked in here where you can use Phoenix to compare the performance of different techniques. 👇 arize.com/blog/prompt-...

⚡️OpenAI Agents Instrumentation is out—we’re continuously improving to bring you the latest. With just a few lines of code, you'll be ready to trace and run agents. Check it out to level up your agent monitoring. Our docs have all the details for a quick setup! 👇 #OpenAI #Agents #Tracing

Some updates for Projects! Gain more flexibility and control with: 📌 Persistent column selection for consistent views 🔍 Filter data directly from tables with metadata and quick metadata filters ⏳ Set custom time ranges for traces & spans 🌳 Option to filter spans by root spans Check out the demo👇

Learn how we built a holistic prompt management system that preserves developer freedom. With Phoenix 8.0, we built a prompt management system that prioritizes: LLM reproducibility, prompt versioning & tracking, & developer flexibility—no vendor lock-in arize.com/blog/prompt-...

Learn how we built a holistic prompt management system that preserves developer freedom. With Phoenix 8.0, we built a prompt management system that prioritizes: LLM reproducibility, prompt versioning & tracking, & developer flexibility—no vendor lock-in arize.com/blog/prompt-...

🧠 Phoenix now supports Anthropic Sonnet 3.7 & Thinking Budgets! This makes Prompt Playground ideal for side-by-side reasoning tests: o3 vs. Anthropic vs. R1. Plus, GPT-4.5 support keeps it up to date with the latest from OpenAI & Anthropic - test them all out in the playground! ⚡️

How can you programmatically improve your prompts? 🤔 🤖 Forget manual prompt engineering - there are better (read: "more automatic") ways to improve your prompts. This video and notebook break down these techniques. Featuring: - DSPy - @arize-phoenix.bsky.social

With Phoenix 8.0 you can version & iterate on prompts seamlessly—both in the UI & in code! What's new: 🔗 TypeScript Client: Sync prompts with your JavaScript runtime 🐍 Python Client: Sync templates & apply them directly to AI SDKs 🔄 Native prompt normalization & much more! youtu.be/qbeohWaRlsM

🤖 Building agents, but not sure how to measure their performance? Our newest blog post on @hf.co has you covered! This post shows you how to use @arize-phoenix.bsky.social to trace and evaluate your smolagents. Credit to @srichavali.bsky.social and @aymeric-roucher.bsky.social

@arize.bsky.social Phoenix 6.0 Prompt Playground! 🛝 📊Test and compare prompts, tool definitions, output schemas, and models, directly in the platform 🎥 Replay spans with adapted prompts 🚄 Run prompts over full datasets 🧪 Automatically trace results as experiments youtu.be/wLK5RwHNLUM?...