How Capital One Delivers Multi-Agent Systems with Rashmi Shetty - #765

What It Covers

Rashmi Shetty, Senior Director of Enterprise Generative AI Platform at Capital One, explains how the company built and deployed Chat Concierge, a multi-agent car-buying system, and outlines the platform strategy enabling developers to build governed agentic systems at scale across the enterprise.

Key Questions Answered

• •Multi-agent trigger criteria: Deploy multi-agent architecture only when a problem contains multiple distinct user intents that cannot be resolved by a single deterministic model. Capital One's Chat Concierge required separate agents for intent disambiguation, planning, governance validation, response accuracy checking, and final response formatting — each with a narrowly scoped task.
• •Risk-first platform layering: Separate agent governance into two distinct layers — platform-level enterprise policies covering cyber, compliance, and guardrails that apply automatically at runtime, and domain-specific policies that individual teams layer on top. This split lets developers focus on agent design while the platform enforces mandatory regulatory boundaries without manual configuration per deployment.
• •Latency as a product feature: Treat end-to-end latency as a first-class product requirement, not a non-functional afterthought. In multi-agent systems, latency must be measured across every agent boundary, tool invocation, and model call simultaneously. Capital One uses smaller specialized fine-tuned models via teacher-student distillation to hit latency targets while maintaining personalization quality.
• •Closed-loop observability design: Instrument agentic systems to capture production failure signals and route them back into the experimentation environment for prompt tuning, model fine-tuning, retrieval adjustment, or context management updates. Design this feedback pipeline before deployment, not after, because production telemetry is where the largest performance gains originate in agentic systems.
• •Beachhead use case selection: Choose the first production agentic deployment from a high-surface-area, low-risk scenario to safely observe real failure modes at scale. Capital One selected an auto dealership customer experience rather than a core banking workflow, generating architectural patterns and observability baselines that informed the broader enterprise platform strategy.