SAP: Bringing the ‘Operating System’ of a Company into the AI Era with CTO Philipp Herzig

What It Covers

SAP CTO Philipp Herzig outlines how the 400,000-customer enterprise software platform is rebuilding its architecture around AI agents, why large language models fail at predictive analytics, and why AI represents a business model transition from seat-based licensing toward consumption and outcome-based pricing models.

Key Questions Answered

• •Enterprise AI adoption gap: The gap between AI innovation and actual enterprise outcomes is widening, not narrowing. Companies with fragmented data from M&A activity or siloed purchasing decisions face the steepest barriers. Organizations that completed data harmonization work over the past decade now adopt AI capabilities measurably faster and achieve return on investment sooner than those starting from scratch.
• •Scale is the hidden engineering problem: Building a RAG chatbot on 10 documents is trivial; scaling to 20,000 APIs creates context bloat and disambiguation failures. SAP's Joule assistant must cross-reference employee location, tax jurisdiction, and payroll data to answer a single travel policy question correctly — a problem that only surfaces when moving from proof-of-concept to production deployment.
• •LLMs cannot replace predictive models for tabular data: Demand forecasting, cash flow prediction, and payment delay classification require classical regression and classification approaches like XGBoost. SAP's two-year NeurIPS-published research produced Relational Pretrained Transformers (RPT-1), a transformer architecture redesigned for structured tabular data that enables high-accuracy predictions from small datasets without dedicated data scientists.
• •Agent mining replaces process mining: As AI agents handle business processes, they surface undocumented "tribal knowledge" — decisions made via phone calls or Slack that never enter systems of record. Capturing agent decision traces allows companies to identify process anomalies, elevate local best practices to global standard operating procedures, and build a data flywheel that continuously improves agent accuracy over time.
• •Pricing is shifting from seats to consumption to outcomes: SAP's current model is a hybrid — primarily seat-based with consumption elements — because enterprise customers still require cost predictability. The trajectory moves toward outcome-based licensing similar to Sierra's model, but only as verifiability of agent outputs improves. Enterprises resisting pure consumption pricing cite fear of runaway costs, not skepticism about AI value.