David Mcintosh

AI Summary

→ WHAT IT COVERS Instacart's Chief Connected Stores Officer David McIntosh explains how the company's Caper Cart — a smart shopping cart powered by NVIDIA Jetson edge AI, sensor fusion, and 1.6 billion historical grocery orders — is digitizing physical retail to unify in-store and online shopping into one personalized experience. → KEY INSIGHTS - **Edge AI over cloud dependency:** Caper Cart runs basket-recognition AI locally on an NVIDIA Jetson board because cloud response times measure in seconds while consumers expect sub-100-millisecond feedback. Sensor fusion combines weight scales, multiple cameras, and location sensors to accurately identify items even when store WiFi drops or cart movement creates ambiguous signals. - **Multi-sensor basket accuracy:** Relying on cameras alone fails in real grocery environments due to variable lighting, cart crowding, and natural shopper movement. The weight sensor functions as a ground-truth "x-ray" of the basket, cross-validated against visual inputs. This combination is essential for checkout accuracy when no store associate is nearby to resolve disputes. - **Personalized "did you forget" feature drives ~1% absolute sales lift:** A checkout-prompt feature surfacing items a specific shopper regularly buys but skipped that trip produced nearly a 1% absolute increase in in-store sales. A subsequent recommendation algorithm update incorporating online delivery signals added another 1% absolute lift on top, demonstrating compounding returns from merging online and in-store data. - **Accurate planogram replacement via shelf-facing cameras:** Most retailers lack accurate store planograms, and layouts vary store-to-store within the same banner. Caper Cart's side-facing cameras continuously scan shelves to determine what is actually stocked and where, feeding a real-time location system that corrects aisle-level ambiguity and prevents irrelevant recommendations that erode user trust over time. - **Grocery foundation model built on 1.6B orders and 2B-item catalog:** Instacart is constructing a grocery-specific foundation model ingesting lifetime delivery orders, a two-billion-item catalog, and in-store behavioral signals — including cart pause locations and item removal patterns. This model underpins agentic applications for shoppers, store associates, and CPG brands, such as automated restocking alerts and shelf-placement optimization recommendations. → NOTABLE MOMENT McIntosh revealed that migrating ad-serving workflows from CPUs to GPUs — announced at GTC — simultaneously reduced latency and increased ad click-through rates in experiments. The result was counterintuitive: a backend infrastructure change produced a measurable consumer-facing behavioral improvement without any change to ad creative or targeting logic. 💼 SPONSORS None detected 🏷️ Smart Cart Technology, Physical AI, Retail Personalization, Edge Computing, Grocery Tech