“Nobody wanted to do this work”: How Emmy Award–winning filmmakers use AI to automate the tedious parts of documentaries

What It Covers

Tim McLear from Ken Burns' Florentine Films uses AI to automate documentary post-production workflows, building custom tools that process hundreds of hours of footage and thousands of images through metadata extraction, embeddings, and semantic search capabilities.

Key Questions Answered

• •Automated metadata generation: Combines OpenAI vision models with embedded file metadata and web scraping to auto-generate accurate descriptions for archival images, reducing manual data entry from hours to seconds while maintaining journalistic accuracy through guardrails that prevent hallucination.
• •Video processing architecture: Extracts frames at five-second intervals using GPT-4o nano for individual captions, pairs with Whisper audio transcription, then sends consolidated data to reasoning models. This multi-step approach balances cost efficiency with comprehensive video analysis for documentary footage databases.
• •Field research iOS app: Custom-built Flip Flop app captures front and back of archival photos, transcribes handwritten notes using OCR, and embeds metadata directly into image EXIF data. This eliminates post-trip file organization chaos and enables 1,400+ images captured per research trip.
• •Semantic discovery through embeddings: Generates dual embeddings using CLIP for image thumbnails and OpenAI text models for descriptions, then fuses them to enable semantic search. This replaces exact keyword matching, allowing editors to find similar portraits or scenes without knowing precise terminology.