20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain

What It Covers

Harrison Chase, founder of LangChain, covers the core challenges of building reliable agents, where value accumulates across the agent stack, how model routing works at harness versus gateway layers, why vertical agents outperform horizontal ones, how LangChain manages 30% of engineering salary on coding tools, and why tier-one VC firms like Benchmark and Sequoia justify taking valuation discounts.

Key Questions Answered

• •Vertical agent architecture: Start building agents at the harness and context layer first, not the model layer. Domain-specific context engineering delivers higher ROI than post-training models early on. Only after accumulating significant user data and usage volume does post-training open-weight models make economic sense. Companies like Harvey and Ramp follow this exact progression, moving from context to harness to model fine-tuning as they scale.
• •Model routing placement: Route models at the harness layer, not the gateway layer, for domain-specific applications. The harness holds more state — prompt cache status, memory, tool availability — enabling smarter routing decisions. Gateway-layer routing (OpenRouter, Vercel) works for horizontal tasks like general chat or coding, but vertical domains like legal or finance require harness-aware routing to achieve meaningful quality and cost optimization.
• •Coding agent cost control — three steps: LangChain reduced coding tool spend while increasing token usage by following three steps: first, pipe all coding agent usage into LangSmith for per-engineer visibility; second, set budget caps requiring VP approval to override; third, build a custom cloud-based coding agent (OpenSUI) with prompt caching and internal routing baked in. This framework is repeatable for any organization managing runaway AI spend.
• •Memory as the unsolved layer: Memory remains the most underdeveloped component in agent systems. The hard part is not the database — it is the intelligence layer that distills observations into storable, retrievable facts. LangChain launched and killed a commercial memory product because general-purpose memory lacks product-market fit. Domain-specific memory agents, built per vertical, represent the viable path forward and will become standard in most products within a few years.
• •Inference market scale: The inference market is significantly larger than most predicted three to four years ago. Demand continues rising as agent-to-agent interactions multiply token consumption exponentially. Multiple inference providers — Fireworks, AWS, Crusoe — can all succeed simultaneously because total demand is large enough to support many players. Organizations should expect inference costs to remain a top-three enterprise concern alongside accuracy and latency for the foreseeable future.
• •Tier-one VC discount calculus: Taking a valuation discount for Benchmark or Sequoia produces measurable returns beyond capital. Benchmark functions as a strategic co-founder focused on company positioning and core product decisions. Sequoia operates as an execution machine — sourcing IC and executive candidates through its talent network, supporting marketing launches, and providing operational infrastructure. Building relationships with target investors two-plus years before a raise reduces the transactional posturing that dominates most fundraising processes.