Inside Instacart's AI-Powered Smart Shopping Cart | NVIDIA AI Podcast Ep. 302

What It Covers

Instacart's Chief Connected Stores Officer David McIntosh explains how the company's Caper Cart — a smart shopping cart powered by NVIDIA Jetson edge AI, sensor fusion, and 1.6 billion historical grocery orders — is digitizing physical retail to unify in-store and online shopping into one personalized experience.

Key Questions Answered

• •Edge AI over cloud dependency: Caper Cart runs basket-recognition AI locally on an NVIDIA Jetson board because cloud response times measure in seconds while consumers expect sub-100-millisecond feedback. Sensor fusion combines weight scales, multiple cameras, and location sensors to accurately identify items even when store WiFi drops or cart movement creates ambiguous signals.
• •Multi-sensor basket accuracy: Relying on cameras alone fails in real grocery environments due to variable lighting, cart crowding, and natural shopper movement. The weight sensor functions as a ground-truth "x-ray" of the basket, cross-validated against visual inputs. This combination is essential for checkout accuracy when no store associate is nearby to resolve disputes.
• •Personalized "did you forget" feature drives ~1% absolute sales lift: A checkout-prompt feature surfacing items a specific shopper regularly buys but skipped that trip produced nearly a 1% absolute increase in in-store sales. A subsequent recommendation algorithm update incorporating online delivery signals added another 1% absolute lift on top, demonstrating compounding returns from merging online and in-store data.
• •Accurate planogram replacement via shelf-facing cameras: Most retailers lack accurate store planograms, and layouts vary store-to-store within the same banner. Caper Cart's side-facing cameras continuously scan shelves to determine what is actually stocked and where, feeding a real-time location system that corrects aisle-level ambiguity and prevents irrelevant recommendations that erode user trust over time.
• •Grocery foundation model built on 1.6B orders and 2B-item catalog: Instacart is constructing a grocery-specific foundation model ingesting lifetime delivery orders, a two-billion-item catalog, and in-store behavioral signals — including cart pause locations and item removal patterns. This model underpins agentic applications for shoppers, store associates, and CPG brands, such as automated restocking alerts and shelf-placement optimization recommendations.