How Foundation Models Evolved: A PhD Journey Through AI's Breakthrough Era

What It Covers

Omar Khattab, creator of DSPy and MIT professor, argues AI needs programmable intelligence through formal abstractions rather than pursuing AGI through scaling alone, introducing signatures as the missing layer between natural language and code.

Key Questions Answered

• •DSPy Signatures: Programs should isolate ambiguity into typed function declarations with named inputs and structured outputs, creating a declarative interface that separates intent from model-specific implementation details, enabling portability across different language models without rewriting prompts.
• •Three Irreducible Components: AI systems require code for control flow and modularity, natural language for fuzzy specifications humans cannot formalize, and data-based optimization for handling edge cases and long-tail problems—no single component alone suffices for expressing complete intent.
• •Scaling Limitations: Frontier labs have abandoned the belief that scaling model parameters and pretraining data alone achieves AGI, now investing heavily in post-training pipelines, retrieval systems, tool use, and agent training to address specification problems rather than pure capability gaps.
• •Optimization Evolution: DSPy optimizers progress from bootstrapping examples with early models to reflective prompt optimization where models debug their own programs, with recent support for online reinforcement learning methods like GRPO applicable to any DSPy program regardless of architecture.