Tim Yerka

AI Summary

→ WHAT IT COVERS LinkedIn VP of Engineering Tim Yerka explains how the platform rebuilt its newsfeed for 1.3 billion members by replacing traditional MLP scoring architectures with sequence-based models and LLM-powered retrieval, covering inference cost management, natural language quality policies, and the engineering team structure required to operate at this scale. → KEY INSIGHTS - **Sequence Modeling for Recommendations:** Treat content recommendation as next-item prediction rather than independent item scoring. LinkedIn models user content paths the way LLMs predict tokens — someone reading an intro to PyTorch article can be mapped 15 steps forward toward building a local LLM. This approach captures contextual momentum that flat feature-crossing architectures like DCNV2 cannot replicate. - **Natural Language Quality Policies:** Define content quality as version-controlled text files checked into GitHub, then use LLM-based auto-evaluation to audit whether surfaced content meets those standards. Product managers iterate on these policy prompts using GitHub Copilot CLI, which returns scored examples and suggests wording revisions — creating a closed feedback loop without requiring manual review at scale. - **GPU Cost Reduction via CPU Offloading and Batching:** Reduce inference costs by routing non-model tasks — language filtering, faceted filtering, preprocessing — to CPU before loading items into GPU memory. Batch feature updates on 20-minute windows rather than per-event writes to reduce database load. Custom CUDA attention kernels further reduce per-request compute cost at the GPU level. - **Multi-Objective Engagement Modeling Personalized Per User:** Optimize simultaneously for positive signals (comments, reshares, dwell time) and negative signals (skips, viewport passes) using a multi-head model. Critically, the weights of these signals are personalized per individual user — not by role or persona — because a job seeker and a marketer require fundamentally different feed optimization targets. - **LLM Context Window Discipline:** Avoid dumping raw structured data into LLM prompts at inference time. LinkedIn compresses member context (profile, recent sessions) and item context (author, post content, image captions) into compact token representations. Poorly constructed context windows degrade precision and inflate compute costs — a dedicated team works specifically on context construction and sequence sampling strategies. → NOTABLE MOMENT Yerka describes a compounding reliability problem with multi-stage agentic systems: if each of 20 sequential steps succeeds 99.5% of the time, the end-to-end workflow becomes effectively unusable. This framing reframes agentic AI reliability as a systems engineering problem requiring grounding signals, not just model accuracy improvements. 💼 SPONSORS [{"name": "XWeather", "url": "https://xweather.com"}, {"name": "WarpBuild", "url": "https://warpbuild.com/sed"}, {"name": "Endor Labs (Auri)", "url": "https://www.endorlabs.com/auri"}] 🏷️ Recommendation Systems, LLM Infrastructure, Inference Optimization, Content Ranking, Responsible AI