470: All about queues with Adam McCrea

What It Covers

Adam McCrea, founder of JudoScale, explains queue optimization strategies for web applications, including SLA-based queue naming, job shape analysis, memory management, and autoscaling approaches that improve performance while reducing infrastructure costs.

Key Questions Answered

• •SLA-Based Queue Naming: Name queues by completion time requirements like "within-5-seconds" or "within-5-minutes" rather than vague names like "high" or "urgent". This creates clear business decisions about job placement and enables precise autoscaling thresholds based on actual service requirements.
• •Queue Latency Over Depth: Monitor how long the oldest job has waited in queue rather than total job count. Latency directly maps to SLA compliance, while depth numbers lack context—a thousand jobs might process in one second or take many minutes depending on job complexity.
• •Three Queue Architecture: Start with three SLA-based queues (five seconds, five minutes, five hours) to handle most use cases. Isolate each queue to separate worker processes for simpler bottleneck identification, independent resource tuning, and the ability to scale memory-intensive queues independently.
• •Memory-Intensive Job Isolation: Quarantine high-memory jobs to dedicated queues with expensive hardware that scales to zero when idle. This prevents a single memory-heavy job from forcing all workers onto costly performance instances, reducing infrastructure costs by 80% in typical scenarios.