Why We Need New AI Benchmarks, Which Industries Survive AI, and Recursive Learning Timelines | #218

What It Covers

Matt Fitzpatrick, CEO of Invisible Technologies and former McKinsey Quantum Black Labs head, explains why enterprises must become AI companies in 2026, covering implementation strategies, custom benchmarks, multi-agent systems, and which industries face disruption versus adaptation.

Key Questions Answered

• •Custom Benchmarks Over General Tests: Enterprises need hyper-specific evaluation frameworks for individual tasks like claims processing or contact center performance, not broad cognitive benchmarks. Companies must build custom evals comparing AI output against expert human performance for their specific workflows and data.
• •Operational Leadership Not IT: Assign best operators, not technology teams, to lead AI initiatives with clear KPIs like CSAT scores, inventory days, or time per call. Locate projects outside IT departments, tie vendor compensation to measurable results, and focus on two to three high-value use cases rather than letting a thousand flowers bloom.
• •Data Preparation Precedes AI: Companies must start with clean, structured data for specific use cases before deploying AI, not attempt to fix entire data lakes. Swiss Gear consolidated 750 data tables to improve inventory forecasting by 30 percent and double reliable SKU predictions within months through targeted data integration.
• •Multi-Agent Architecture Dominates: Successful enterprise AI uses task-specific agents orchestrated by large language models, not single all-purpose agents. This architecture enables pinpoint accuracy on individual functions while maintaining coordination, as demonstrated in contact centers where human-AI hybrid models outperform fully autonomous systems like Klarna's failed rollout.
• •Human Expertise Remains Essential: Industries requiring physical work, human interaction, or decisions without precedent data will maintain human roles. Legal services, real estate evaluation, and sales relationships persist while commodity documentation and basic information lookup tasks face automation. Twenty-five percent of workers enter fields that did not exist during their education.