Building Durable AI Agents
Episode
46 min
Read time
2 min
Topics
Startups, Artificial Intelligence, Software Development
AI-Generated Summary
Key Takeaways
- ✓Agent Harness Architecture: The harness — the software program that maps LLM token outputs to actual tool calls and actions — is distinct from the model itself. Anthropic's Claude Opus 4.8 and Claude Code have become tightly coupled through reinforcement learning, meaning swapping in GPT-5.5 into the Claude Code harness produces measurably worse tool-calling accuracy and task performance.
- ✓Production Infrastructure Pattern: Deploying agents beyond a local machine requires a message broker between the API entry point and workers. A FastAPI server places events on a durable message queue; workers spin up independently to process agentic loops. This prevents total failure when individual workers go down due to network issues or compute contention at scale.
- ✓Checkpoint Everything First: Before optimizing agent performance, instrument every tool call and LLM interaction as a checkpoint stored in an external database or blob storage. After one week of production runs, filter for the most expensive successful traces, identify common bottlenecks, and address failure modes — rather than writing defensive code that slows development velocity.
- ✓Model Supply Chain Risk: Enterprises building agents on proprietary model providers face operational risk if those models become unavailable. Open-source models like GLM-4 now reach approximately 95% of Claude Opus 4.8 performance, making internal agent platforms built on open harnesses like LangGraph or Pydantic AI a viable strategy for avoiding single-provider dependency.
- ✓Replay-Based Experimentation: Kitaru enables replaying production agent traces with swapped models or modified tool sets to evaluate cost and quality tradeoffs without re-running full workflows from scratch. A key limitation: replacing a model mid-trace produces a broken experiment because the agent may never have reached that state with the substitute model from the start.
What It Covers
Hamza Tahir, cofounder of ZenML and creator of Kitaru, explains why AI agents running in cloud environments fail at scale, how MLOps principles apply to agentic systems, and what infrastructure patterns — checkpointing, task queues, replay — make agents durable in production enterprise deployments.
Key Questions Answered
- •Agent Harness Architecture: The harness — the software program that maps LLM token outputs to actual tool calls and actions — is distinct from the model itself. Anthropic's Claude Opus 4.8 and Claude Code have become tightly coupled through reinforcement learning, meaning swapping in GPT-5.5 into the Claude Code harness produces measurably worse tool-calling accuracy and task performance.
- •Production Infrastructure Pattern: Deploying agents beyond a local machine requires a message broker between the API entry point and workers. A FastAPI server places events on a durable message queue; workers spin up independently to process agentic loops. This prevents total failure when individual workers go down due to network issues or compute contention at scale.
- •Checkpoint Everything First: Before optimizing agent performance, instrument every tool call and LLM interaction as a checkpoint stored in an external database or blob storage. After one week of production runs, filter for the most expensive successful traces, identify common bottlenecks, and address failure modes — rather than writing defensive code that slows development velocity.
- •Model Supply Chain Risk: Enterprises building agents on proprietary model providers face operational risk if those models become unavailable. Open-source models like GLM-4 now reach approximately 95% of Claude Opus 4.8 performance, making internal agent platforms built on open harnesses like LangGraph or Pydantic AI a viable strategy for avoiding single-provider dependency.
- •Replay-Based Experimentation: Kitaru enables replaying production agent traces with swapped models or modified tool sets to evaluate cost and quality tradeoffs without re-running full workflows from scratch. A key limitation: replacing a model mid-trace produces a broken experiment because the agent may never have reached that state with the substitute model from the start.
Notable Moment
Tahir describes updating a production agent as genuinely terrifying — even adding a single word to a system prompt can produce unpredictable outcomes across hundreds of millions of in-flight enterprise executions, revealing that agent versioning and safe deployment remain largely unsolved problems in 2025.
Episode Transcript
Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, x, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. Now onto the show. Welcome to another episode of the Practical AI podcast. This is Daniel Whitenack. I am CEO at Prediction Guard and really excited for today's episode because it fits right in the theme of our show, which is Practical AI, focusing on some things that are actually useful and practical. Have with us today Hamza Tahir, who is cofounder at Zen ZenML. And they they have a new, product, a new project, Al Kitaru, who who, which is focused on agents and making agents durable, which is is super interesting. And Hamza is joining us, today. I I think you were out at the AI engineers world's fair. Right? Yeah. I am. It's, like, 7,000 people. All of our crowd gathered in one small like, big hallway. So it's it's just fantastic to be in San Francisco when the energy is so high. Yeah. That's that's awesome. Always always inspiring and and really cool to see also growth in that from Swyx and and others who who've really built up an amazing community over time. Friend friends of the friends of the show. So if you haven't checked it out, go ahead and and check out what they're doing over there. But, yeah, ex excited to dig in today, Hamza. May maybe just, to to set the stage, I know, your cofounder of ZenML is kind of some background with that project and product around ML ops. Now you're getting into agent agentic, things. I I love your perspective on maybe first off kind of the the world that you have been inhabiting around ML and ML pipelines. As now we're all thinking about agents and generative AI and all of these things, like, what from your perspective, before we get into agents specifically, like, what role does the more traditional ML, models, training pipelines, etcetera, play in in our world moving moving forward from your perspective? Awesome. That's, I think, a great one to start with. And thank you for for, like, inviting me on the show. Yeah. I appreciate the opportunity. I so I cofounded ZenML about five years ago. So this was really almost, at a point where MLOps was really reaching fever pitch on, you know, but there was all sorts of chatter about how to productionize AI and machine learning workloads. And I had done four or five years of that in my previous job where I …
Get the full transcript (8,446 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 43-minute episode.
Get Practical AI summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Practical AI
Models, Harnesses, and Multi-Agent Systems
Aug 6 · 49 min
Latent Space
Simulation: the new Scaling Law — Joon Sung Park, Simile AI
Aug 21
More from Practical AI
Reconstructing how OpenAI agents attacked Hugging Face
Jul 30 · 44 min
Eye on AI
Every Enterprise Is About to Have a 100,000 Agent Problem | Oren Michaels of Barndoor AI
Jun 6
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.
Tools
- KitaruBy guest
by Hamza Tahir
“Hamza Tahir, cofounder of ZenML and creator of Kitaru, explains why AI agents running in cloud environments fail at scale, how MLOps principles apply to agentic systems, and what infrastructure patterns — checkpointing, task queues, replay — make agents durable in production enterprise deployments.”
“A FastAPI server places events on a durable message queue; workers spin up independently to process agentic loops.”
“Open-source models like GLM-4 now reach approximately 95% of Claude Opus 4.8 performance, making internal agent platforms built on open harnesses like LangGraph or Pydantic AI a viable strategy for avoiding single-provider dependency.”
“Open-source models like GLM-4 now reach approximately 95% of Claude Opus 4.8 performance, making internal agent platforms built on open harnesses like LangGraph or Pydantic AI a viable strategy for avoiding single-provider dependency.”
Products
by Anthropic
“Anthropic's Claude Opus 4.8 and Claude Code have become tightly coupled through reinforcement learning, meaning swapping in GPT-5.5 into the Claude Code harness produces measurably worse tool-calling accuracy and task performance.”
by Anthropic
“Anthropic's Claude Opus 4.8 and Claude Code have become tightly coupled through reinforcement learning, meaning swapping in GPT-5.5 into the Claude Code harness produces measurably worse tool-calling accuracy and task performance.”
company
- ZenMLBy guest
“Hamza Tahir, cofounder of ZenML and creator of Kitaru, explains why AI agents running in cloud environments fail at scale”
More from Practical AI
We summarize every new episode. Want them in your inbox?
Models, Harnesses, and Multi-Agent Systems
Reconstructing how OpenAI agents attacked Hugging Face
Surviving the New Economics of a Post-Agentic World
The Future of AI Infrastructure with CoreWeave
Image Generation and Visual Intelligence with Black Forest Labs
Similar Episodes
Related episodes from other podcasts
Latent Space
Aug 21
Simulation: the new Scaling Law — Joon Sung Park, Simile AI
Eye on AI
Jun 6
Every Enterprise Is About to Have a 100,000 Agent Problem | Oren Michaels of Barndoor AI
The TWIML AI Podcast
May 7
How to Find the Agent Failures Your Evals Miss with Scott Clark - #767
Cognitive Revolution
May 6
"Descript Isn't a Slop Machine": Laura Burkhauser on the AI Tools Creators Love and Hate
NVIDIA AI Podcast
Feb 4
How AI-Powered Holograms Are Reimagining Fan Experiences at the Big Game - Ep. 288
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Practical AI.
Every Monday, we deliver AI summaries of the latest episodes from Practical AI and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime