The Mathematical Foundations of Intelligence [Professor Yi Ma]

What It Covers

Professor Yi Ma presents a mathematical theory of intelligence based on parsimony and self-consistency principles, explaining how compression drives knowledge acquisition across evolutionary, neural, and scientific stages while deriving white-box transformer architectures from first principles.

Key Questions Answered

• •Rate Reduction Framework: Intelligence operates by discovering low-dimensional structures in high-dimensional data through compression, where the coding rate measures data volume. This principle explains memory formation across DNA evolution, neural learning, and scientific discovery as fundamentally the same compression process with different mechanisms.
• •White-Box Transformers (CRATE): Multi-head self-attention emerges mathematically as gradient steps optimizing rate reduction objectives, while MLPs function as sparsification operators. This derivation eliminates dozens of hyperparameters and achieves linear time complexity versus quadratic in standard transformers, enabling principled architecture design rather than empirical search.
• •Compression vs Abstraction Distinction: Current large language models memorize text distributions through empirical compression mechanisms but lack the phase transition to abstraction that enables deductive reasoning. Understanding requires moving beyond statistical correlation extraction to formalized logical structures, representing a fundamental gap in artificial intelligence capabilities.
• •Self-Consistent Learning Loop: Autonomous learning requires closed-loop prediction and correction within the brain rather than end-to-end supervision. When data distributions have sufficient low-dimensional structure, systems can minimize reconstruction error internally through perception channels alone, enabling continual learning without external ground truth measurement.
• •Benign Optimization Landscapes: Natural low-dimensional structures create highly regular, symmetric loss surfaces with no spurious local minima or flat regions. This blessing of dimensionality explains why gradient descent succeeds in deep learning and why intelligence naturally identifies easy-to-learn patterns first, contradicting worst-case complexity theory assumptions.