#496 – FFmpeg: The Incredible Technology Behind Video on the Internet

What It Covers

Lex Fridman speaks with Jean-Baptiste Kempf, president of VideoLAN, and Kieran Clunya, FFmpeg contributor, about the open-source software stack powering video across the internet. FFmpeg processes over 90% of online video workflows. VLC has been downloaded 6.5 billion times. Both projects run on volunteer labor, with core teams of 5–15 people maintaining infrastructure used by billions daily.

Key Questions Answered

• •Video compression ratios: Modern video codecs achieve 1,000x compression by exploiting human perceptual limits rather than lossless data reduction. Codecs operate in YUV color space instead of RGB, immediately halving file size because the human eye is more sensitive to brightness than color. Each new codec generation delivers roughly 30% better quality at the same bitrate, but requires an order of magnitude more CPU power to encode — making compression asymmetric by design.
• •Open-source licensing as community contract: Choosing between MIT, GPL, LGPL, and Apache licenses is not a legal formality — it defines the social structure of a contributor community. LGPL allows commercial applications to embed libraries like LibVLC or FFmpeg without open-sourcing their entire product, enabling consulting businesses to form around the technology. GPL requires full source disclosure, which can block App Store distribution. Relicensing requires tracking down every contributor individually, including deceased ones.
• •Volunteer sustainability gap: FFmpeg's core is maintained by 10–15 volunteers while running on an estimated hundreds of millions to one billion CPU cores simultaneously across platforms including YouTube, Netflix, Chrome, and Discord. Trillion-dollar corporations file high-priority bug reports on public trackers without financial contribution. A single Microsoft Teams engineer labeled an FFmpeg issue high priority by invoking the Microsoft brand name, prompting the project to publicly request a support contract — which was declined in favor of a one-time payment of a few thousand dollars.
• •Security report inflation problem: AI-generated security vulnerability reports from large organizations create a denial-of-service effect on volunteer maintainers. Reports are uniformly marked high-severity regardless of real-world exploitability — including a case where a single pixel could render the wrong color, rated 7.5 severity in red. A separate report targeted an obscure 1990s game codec present on one disc. A 16-year-old contributor found and fixed a genuine issue in three days without filing a public CVE or creating media coverage.
• •Assembly programming as competitive advantage: FFmpeg's performance at billion-CPU scale depends on hand-written assembly optimized for specific CPU architectures, including SIMD instruction sets and pipeline timing measured in individual clock cycles. Teenagers have contributed thousands of lines of production assembly to FFmpeg. Understanding CPU architecture — pipelining, ALU behavior, IO — is the skill gap between average software engineers and the contributors who maintain this infrastructure. Andrew Kelly, creator of the Zig programming language, developed his skills through FFmpeg contributions.
• •VLC's resilience-first architecture: VLC was originally built to play MPEG-2 streams over UDP networks, where packet loss is expected. This forced a design philosophy of never trusting input data — the system attempts playback regardless of file corruption, missing metadata, or mismatched containers. This is why VLC could play partially downloaded files during the peer-to-peer era when metadata stored at the end of AVI files was unavailable. File extensions are treated as low-priority hints, not authoritative format declarations.
• •Turning down monetization to preserve trust: JB received multiple acquisition and partnership offers — including one described as obscene — to bundle spyware toolbars, change default search engines, or insert advertisements into VLC. Each offer was declined. The reasoning: accepting would have generated short-term revenue but triggered a community fork within three years, destroying the project. The only leverage a small open-source team holds against large platforms — Google Play Store, Apple App Store, Windows Store — is the credible threat of pulling distribution of software used by hundreds of millions of users.