Write, Change, Recall, Forget: MongoDB's Pete Johnson on How Retrieval Drives Agent Performance

What It Covers

MongoDB Field CTO Pete Johnson traces database evolution from SQL's 1970 origins through MongoDB's 2007 founding to today's AI agent infrastructure. The conversation covers vector search architecture, hybrid retrieval methods, the Voyage AI acquisition, enterprise memory system design patterns, and why agent performance and cost optimization depend fundamentally on retrieval quality rather than context window maximization.

Key Questions Answered

• •Retrieval over context stuffing: Naively filling million-token context windows costs multiple dollars per call and degrades answer quality. Academic research shows only the first and last ~7,000 tokens carry significant weight, with middle content confusing models. Enterprises now prioritize selecting the right 200,000 tokens per agentic loop rather than maximizing context. Effective retrieval via vector search, lexical search, and pre-filtering combined is the primary lever for improving both cost-adjusted agent performance and answer accuracy.
• •Embeddings are not commoditized: Developers commonly assume embedding models are interchangeable, but Voyage AI models rank at the top of Hugging Face's MTEB benchmark, delivering up to 14% retrieval quality improvement over common alternatives like OpenAI or Gemini embeddings. Adding a re-ranker on top of any embedding model yields an additional 5–10% retrieval quality boost. Anthropic has no embedding model and actively recommends Voyage. Embedding model selection is a concrete, high-leverage decision that directly affects hallucination rates.
• •Hybrid search in a single API call: MongoDB's rank fusion and score fusion aggregation pipeline stages allow developers to combine vector search, lexical search, and metadata pre-filtering in one round-trip server call. Previously, developers had to execute searches separately and merge results manually. The dollar re-rank stage further integrates re-ranking into the same pipeline. This architectural consolidation reduces latency, eliminates custom merging code, and lowers maintenance burden for production RAG pipelines.
• •Contextualized chunking eliminates chunk-size iteration: Traditional embedding models require developers to iterate through multiple chunk sizes to balance retrieval quality against storage cost, since smaller chunks lose context and larger chunks degrade precision. Voyage AI's contextualized chunking accepts both the target sentence and surrounding context as separate inputs, returning a single vector array that encodes both. This inverts the normal trade-off, achieving higher retrieval quality at smaller chunk sizes without manual iteration cycles.
• •Write-change-recall-forget memory architecture: Effective agent memory systems require two developer responsibilities: querying memory with a token budget to retrieve the most relevant context across short-term, long-term, and taxonomic memory types, then returning the agent's output back to the memory system for curation and storage. Forgetting is the hardest unsolved problem. Taxonomic memory, which stores domain-specific terminology and retrieves only the five or so relevant terms per agentic loop rather than all hundred, is emerging as a distinct memory type in Fortune 500 deployments.
• •Problem selection determines AI ROI: Enterprises stuck in POC purgatory typically chose problems without pre-existing performance metrics, making ROI attribution impossible. The framework for selecting high-value AI problems requires three criteria: the problem ranks among the top 10–15 business priorities, the organization already has clean data for it, and current performance metrics exist before AI is introduced. Call centers and software delivery lifecycle are cited as reliable starting points because cost-per-call and idea-to-deployment time are already tracked.
• •Shared embedding spaces reduce development token costs: Voyage AI's four text model variants (nano, small, medium, large) share a single embedding space, meaning vectors generated by one model are compatible with the others. Developers can embed their full corpus using the large model while running all development queries through the open-weight nano model, which runs locally at zero token cost. This eliminates embedding API costs during the development and testing phase, though a modest retrieval quality trade-off applies when mixing model sizes.