AI Summary
→ WHAT IT COVERS Hamza Tahir, cofounder of ZenML and creator of Kitaru, explains why AI agents running in cloud environments fail at scale, how MLOps principles apply to agentic systems, and what infrastructure patterns — checkpointing, task queues, replay — make agents durable in production enterprise deployments. → KEY INSIGHTS - **Agent Harness Architecture:** The harness — the software program that maps LLM token outputs to actual tool calls and actions — is distinct from the model itself. Anthropic's Claude Opus 4.8 and Claude Code have become tightly coupled through reinforcement learning, meaning swapping in GPT-5.5 into the Claude Code harness produces measurably worse tool-calling accuracy and task performance. - **Production Infrastructure Pattern:** Deploying agents beyond a local machine requires a message broker between the API entry point and workers. A FastAPI server places events on a durable message queue; workers spin up independently to process agentic loops. This prevents total failure when individual workers go down due to network issues or compute contention at scale. - **Checkpoint Everything First:** Before optimizing agent performance, instrument every tool call and LLM interaction as a checkpoint stored in an external database or blob storage. After one week of production runs, filter for the most expensive successful traces, identify common bottlenecks, and address failure modes — rather than writing defensive code that slows development velocity. - **Model Supply Chain Risk:** Enterprises building agents on proprietary model providers face operational risk if those models become unavailable. Open-source models like GLM-4 now reach approximately 95% of Claude Opus 4.8 performance, making internal agent platforms built on open harnesses like LangGraph or Pydantic AI a viable strategy for avoiding single-provider dependency. - **Replay-Based Experimentation:** Kitaru enables replaying production agent traces with swapped models or modified tool sets to evaluate cost and quality tradeoffs without re-running full workflows from scratch. A key limitation: replacing a model mid-trace produces a broken experiment because the agent may never have reached that state with the substitute model from the start. → NOTABLE MOMENT Tahir describes updating a production agent as genuinely terrifying — even adding a single word to a system prompt can produce unpredictable outcomes across hundreds of millions of in-flight enterprise executions, revealing that agent versioning and safe deployment remain largely unsolved problems in 2025. 💼 SPONSORS [{"name": "Framer", "url": "https://framer.com/practicalai"}] 🏷️ AI Agents, MLOps, Agent Infrastructure, Open Source Models, Production AI