Arize AI

🎟️ Tickets are LIVE for Arize:Observe! 🎟️ The premier event for AI engineers, researchers, and industry leaders is back. Join us in San Francisco on June 25 at SHACK15 for a full day of insights, discussions, and networking—all focused on AI evaluation, observability, and the next generation of agents and assistants. ✔️ Learn from experts tackling AI's biggest challenges ✔️ Explore cutting-edge techniques for evaluating AI agents & assistants ✔️ Connect with industry leaders shaping the future of AI As AI systems become more autonomous and high-stakes, staying ahead with rigorous evaluation methods is essential. Don’t miss this deep dive into the future of AI observability. 🎪 Get your tickets: arize.com/observe-2025

📢 Just updated: New speakers + sessions added to Wednesday’s SF Builders Meetup. We’ve added two solid tech talks focused on what actually matters when scaling AI agents—from evaluation to flywheels to real-world deployment. Here’s what’s new on the agenda: Sylendran Arunagiri (NVIDIA) will dig into NeMo microservices, continuous tuning, and how to keep agents aligned as real-world conditions change. You'll get a sneek peek at  NVInfobot, NVIDIA’s internal AI agent powered by its own data flywheel. Srilakshmi Chavali (Arize) will share a practical framework for building agents that improve in production—covering routing, memory, skill selection, and eval strategies that go beyond static tests. Plus: ✖️ Community demos (engineers building real systems) ✖️ Snacks, drinks, and a good excuse to hang out with people solving similar problems 📍 SF | Wednesday @ 5:30pm Register here:  https://lnkd.in/dHXhc2zb

Observability integrated directly into CrewAI agents 🚀 CrewAI lets you orchestrate autonomous multi-agent systems to tackle complex tasks across tools, APIs, and models With native support for both Arize AX and Arize Phoenix, you can now trace, evaluate, and debug your CrewAI agents with no manual instrumentation. As agents reason, delegate, and execute—every LLM call, tool invocation, and decision point is automatically captured Traces include full metadata: role, task, input/output, step sequence, and downstream calls. With this integration, you can: 📡 Trace and visualize agent workflows across chains and tools 📊 Run structured evals on each step (e.g. LLM-as-a-judge) 🧠 Identify breakdowns in reasoning or tool usage ⚙️ Maintain consistent monitoring across agents and providers Integration guide for Arize: Arize Documentation ➡️ https://lnkd.in/epcJg55f Integration guide for Phoenix: Phoenix Documentation ➡️ https://lnkd.in/eAYv9rGZ

We're back for another Builders Meetup at GitHub HQ in SF this week, and this time we're teaming up with the incredible folks at NVIDIA. 🔥 Get ready to dive into the world of automated agent improvement. We'll be tackling big questions together: How can you construct a system out of the building blocks of tracing, evaluation, experiments, monitoring, and prompt engineer/optimization that autonomously improves your agent? Is vibe-optimization the next iteration of vibe-coding? Does this approach even work? Join us for insightful talks by experts at Arize & NVIDIA, plus inspiring community demos (see what other people are building), and built-in networking opportunities. Catch up with old friends and make new ones. ✌ Register: https://lnkd.in/dHXhc2zb

Building LLM agents is one thing — understanding how they behave is a whole different challenge. In our latest tutorial, we show you how the OpenAI Agents SDK and Arize Phoenix work together to help you evaluate, debug, and improve agent performance with ease: 1. Trace every decision your agent makes 2. Run structured experiments on benchmark datasets 3. Evaluate with LLM-as-a-judge 4. Monitor and debug agents with online evaluations in production If you want to ensure your agents are performing at their best, start by understanding how they work 🚀 Check out the full video: https://lnkd.in/eadq7_Fd Cookbook: https://lnkd.in/eWUTgQKa

As AI systems get more powerful, keeping their outputs grounded in truth is a big challenge. We’re excited to have Vectara in the mix at Observe—bringing real expertise in making GenAI more reliable. 🚀 Vectara is changing how we build AI Assistants and Agents—making them accurate, safe, and actually grounded in your own data. That mission fits right in with our focus on evaluating and improving AI systems as they scale. Join the builders, researchers, and leaders tackling one of AI’s most urgent questions: How do we ensure results and reliability as these systems grow more complex? ( 👀 See who'll be answering that here: https://lnkd.in/grhcs8EW ) With AI agents now making high-stakes decisions across industries, developing smarter ways to evaluate, monitor, and improve them has never been more important. Join us June 25 in SF 👉 https://lnkd.in/e9bnYnFa

Arize Observe Speaker Spotlight: Lorenze Jay Hernandez (CrewAI) ✨ Join us June 25th in San Francisco for a full-day event focused on ensuring results and reliability in today's evolving AI landscape. We're bringing together practitioners who are actively solving these challenges so they can build the next generation of AI systems. Expect in-depth conversations on: ✔️ Developing and evaluating advanced AI agents ✔️ Building trust and confidence in autonomous systems ✔️ Addressing considerations for responsible & reliable AI deployment ✔️ Real-world applications and future trends in AI Stay tuned as we reveal more of our speaker lineup and the topics they'll exploring. Don't miss your chance to connect with the pioneers shaping the future of AI. Register or see the speaker lineup (so far) 👉 https://lnkd.in/gPxyfCtn

🚀 Phoenix now supports MCP (Model Context Protocol) This lets tools like Claude query prompts, datasets, and experiment results directly from a Phoenix instance (cloud or self-hosted) Check out our docs to learn more about how to spin it up 👇 Here’s how it works: Arize AI

Trace Your Flowise LLM Apps with Arize Phoenix 🔍 Building LLM applications with FlowiseAI (YC S23) is fast and intuitive with its low-code, drag-and-drop interface. Now, with the Arize Phoenix integration, you can go beyond building and start understanding how your apps behave under the hood. As you develop agents and flows, Phoenix gives you visibility into every step: 🔁 Trace LLM calls 🔀 Inspect router logic 🛠️ Debug tool executions 🧠 Visualize full agent workflows With just 1 configuration step, you can start capturing rich observability data — no code required. Dive into our docs to get started: https://lnkd.in/efvnHxUd

Get ready to learn from the best! Want to know how teams at OpenAI, Jasper, Figma, and Tripadvisor are ensuring results and reliability as systems grow more complex, powerful, and autonomous? Join us at Arize Observe to find out. ✨ We're excited to announce a few speakers here from our first round: Rahul Todkar, Melody Meckfessel, Lukas Gross & Rodrigo Davies. 🙌 Plus more great folks on the site: https://lnkd.in/e9bnYnFa On June 25, industry leaders, builders, and researchers will tackle AI's biggest challenge: ensuring results and reliability in increasingly complex and autonomous systems. With AI agents making high-stakes decisions across industries, new frameworks to evaluate, monitor, and improve these systems have never been more critical. Don't miss out on this opportunity to gain actionable insight from people on the frontlines. We'll see you in June at SHACK15 in San Francisco. 🫡 📣 PSA: You have about 24 hours until prices go up. Get tickets: https://lnkd.in/gPxyfCtn #ArizeObserve #AI #AIEvaluation #LLM #AIObservability