404: The Transcription Challenge: Building Infrastructure That Scales With The World

What It Covers

Arvid Kahl explains how he built PodScan's transcription infrastructure to process 50,000 podcast episodes daily, reducing costs from potential $100,000 monthly to just $2,000 through strategic GPU selection and optimization techniques.

Key Questions Answered

• •GPU Selection Strategy: Smaller RTX 4000 GPUs at €200 monthly outperform expensive H100s for transcription when measured by words-per-dollar ratio. Running 10 Hetzner servers with modest GPUs costs $2,000 monthly versus $30,000 for premium AI-focused hosting services.
• •Memory Management Trade-offs: Limiting parallel transcription processes to 2-3 per GPU instead of maxing out VRAM capacity prevents quality degradation and hallucinations. Full GPU utilization causes competing processes to produce unreliable transcripts when memory limits are reached, making conservative allocation essential.
• •Diarization Prioritization System: Speaker detection consumes twice the processing time of transcription itself. Disabling diarization for single-speaker shows doubles daily transcription capacity, allowing resources to process historical episodes while maintaining real-time coverage of 50,000 new daily releases.
• •Database Architecture Scaling: Storing transcripts directly in MySQL becomes unmanageable beyond initial scale. Moving transcripts older than months to S3 storage as JSON files and using OpenSearch clusters for full-text queries prevents database bloat and maintains query performance at multi-terabyte scale.