Tracing and telemetry traditionally has been an operational requirement, not a development one. But I've found that with AI applications this fundamentally changes.

🐳 @arize.bsky.social OSS Prompt Playground @arize-phoenix.bsky.social gets Deepseek support! Now you can compare outputs of all the top tier reasoning models. Which LLM provider would you like to see next? Let us know on GitHub! github.com/Arize-ai/pho...

👨‍🍳 @arize-phoenix.bsky.social continues to cook Announcing OpenInference instrumentation for Agno, Mastra, Bedrock Agents, and AutoGen AgentChat! At @arize.bsky.social we believe observability deserves to be built in the open s/o @anthonypowell.me and many others github.com/Arize-ai/ope...

🧪 📊 The @arize-phoenix.bsky.social TS/JS client now supports Experiments and Datasets! You can now create datasets, run experiments, and attach evaluations to experiments using the Phoenix TS/JS client. Shoutout to @anthonypowell.me and @mikeldking.bsky.social for the work here!

@arizeai/[email protected] - @arize-phoenix.bsky.social javascript client gets experiments 🧪 s/o @anthonypowell.me ! - native tracing of ai tasks and evaluators, - async concurrency queues - support for any evaluator (e.g. bring your own evals) and more!

OpenTelemetry instrumentation for Agno is published! Huge s/o to Dirk Brand. A true testament that AI observability should be built in the open 👏 @arize-phoenix.bsky.social pypi.org/project/open...

📝Annotation Configs in @arize-phoenix.bsky.social Part of the "Look at the Data" initiative, create custom rubrics and forms to annotate your spans. s/o to @anthonypowell.me here who built out all the rich UI features.

9⃣ @arize-phoenix.bsky.social is gonna turn 9 today. Project Retention Policies Customize the data retention of your projects by number of days or by trace count. No more cron jobs or manual deleting of traces needed! A much requested ask from our on-prem users and phoenix-cloud users alike.

Learn to prompt better

I'll be speaking at Arize:Observe at SHACK15 on June 25! Looking forward to exploring what’s next for AI agents & assistants. More details on my session to come. @arize.bsky.social arize.com/observe-2025

I still own plenty of pencils but no erasers. What does that say about me?

Just dropped a tutorial on using the OpenAI Agents SDK + @arize-phoenix.bsky.social to go from building to evaluating agents. ✔️ Trace agent decisions at every step ✔️ Offline and Online Evals using LLM as a Judge If you're building agents, measuring them is essential. Full vid and cookbook below

Demo your app at this year's Observe! Fill out a short application by 4.30 to be considered for our Demo Den. Great opportunity to showcase your work to the AI community in SF. Apply here: docs.google.com/forms/d/e/1F...

"The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise." - Edsger W. Dijkstra. Just read this and I am going to be using it a LOT.

In case you missed it, Arize AI Phoenix crossed the 5k GitHub star mark last week! ⭐️ Phoenix has changed a TON since its first iteration. I'm constantly in awe of the execution speed and quality of this team. Here's to the next 5k and beyond!

Love the community we're building!

For all my NYC friends! 🗽🍎 We're hosting an in-person office hours tomorrow all around LLM and Agent Evals. Join for the free snacks/drinks, stay for the heated discussions about the validity of Pokemon-based model evaluations ⚡️🐀

How much more data does an LLM app really need? In my latest tutorial, I explore how few-shot prompting boosts accuracy without massive datasets or retraining—using @arize-phoenix.bsky.social prompts and experiments to break it down. This kicks off my prompting series... more to come!

🤖 OpenAI 's agent framework openai-agents provides a rich set of composable primitives that enable you to build agents. openinference-instrumentation-openai-agents, an OpenTelememetry instrumentor that is compatible with any OTel backend like @arize-phoenix.bsky.social. Fully OSS and free to use!

How can you programmatically improve your prompts? 🤔 🤖 Forget manual prompt engineering - there are better (read: "more automatic") ways to improve your prompts. This video and notebook break down these techniques. Featuring: - DSPy - @arize-phoenix.bsky.social

Learn how we built a holistic prompt management system that preserves developer freedom. With Phoenix 8.0, we built a prompt management system that prioritizes: LLM reproducibility, prompt versioning & tracking, & developer flexibility—no vendor lock-in arize.com/blog/prompt-...

AI is all about vibes lately

With Phoenix 8.0 you can version & iterate on prompts seamlessly—both in the UI & in code! What's new: 🔗 TypeScript Client: Sync prompts with your JavaScript runtime 🐍 Python Client: Sync templates & apply them directly to AI SDKs 🔄 Native prompt normalization & much more! youtu.be/qbeohWaRlsM

🤖 Building agents, but not sure how to measure their performance? Our newest blog post on @hf.co has you covered! This post shows you how to use @arize-phoenix.bsky.social to trace and evaluate your smolagents. Credit to @srichavali.bsky.social and @aymeric-roucher.bsky.social

AI agents are just tools. They're not engineers. They're not people. While I get the value of describing AI agents as human, I think it's a tad lazy and it creates a self-sabotaging narrative for our industry. dev.to/blackgirlbyt...

Learning Cursor and Vim at the same time was a bad idea. They philosophically clash. One is for vibe coding driven by AI instinct, the other is vibe coding driven by human instinct. Switching to Zed for a while to focus on building my own intuitions.

AI is deeply political

New theme @anthonypowell.me ? cs16.samke.me

From Skeptic to Advocate: How I Became an Open-Source Engineer mikeldking.github.io/2025/01/05/b...

What is the ideal structure for running a great dev relations team? I somehow picture it like a newspaper in a Hollywood movie… people pitching ideas and the best one winning. Am I wrong?

Prompt Playground is out in @arize.bsky.social Phoenix 6.0 📊Test and compare prompts, tool definitions, output schemas, and models, directly in the platform 🎥 Replay spans with adapted prompts 🚄 Run prompts over full datasets 🧪 Automatically trace results as experiments github.com/Arize-ai/pho...