Skip to main content
HF

How Foundation Models Evolved

1episode
1podcast

We have 1 summarized appearance for How Foundation Models Evolved so far. Browse all podcasts to discover more episodes.

Featured On 1 Podcast

All Appearances

1 episode

AI Summary

→ WHAT IT COVERS Omar Khattab, creator of DSPy and MIT professor, argues AI needs programmable intelligence through formal abstractions rather than pursuing AGI through scaling alone, introducing signatures as the missing layer between natural language and code. → KEY INSIGHTS - **DSPy Signatures:** Programs should isolate ambiguity into typed function declarations with named inputs and structured outputs, creating a declarative interface that separates intent from model-specific implementation details, enabling portability across different language models without rewriting prompts. - **Three Irreducible Components:** AI systems require code for control flow and modularity, natural language for fuzzy specifications humans cannot formalize, and data-based optimization for handling edge cases and long-tail problems—no single component alone suffices for expressing complete intent. - **Scaling Limitations:** Frontier labs have abandoned the belief that scaling model parameters and pretraining data alone achieves AGI, now investing heavily in post-training pipelines, retrieval systems, tool use, and agent training to address specification problems rather than pure capability gaps. - **Optimization Evolution:** DSPy optimizers progress from bootstrapping examples with early models to reflective prompt optimization where models debug their own programs, with recent support for online reinforcement learning methods like GRPO applicable to any DSPy program regardless of architecture. → NOTABLE MOMENT Khattab reframes the goal from artificial general intelligence to artificial programmable intelligence, comparing the need for both chatbot agents and formal systems to needing both chairs and tables at home—different tools serve fundamentally different purposes in software development. 💼 SPONSORS None detected 🏷️ DSPy Framework, AI Abstractions, Foundation Models, Prompt Optimization

Explore More

Never miss How Foundation Models Evolved's insights

Subscribe to get AI-powered summaries of How Foundation Models Evolved's podcast appearances delivered to your inbox weekly.

Start Free Today

No credit card required • Free tier available