[State of Post-Training] From GPT-4.1 to 5.1: RLVR, Agent & Token Efficiency — Josh McGrath, OpenAI

What It Covers

Josh McGrath from OpenAI discusses post-training evolution from GPT-4.1 to 5.1, covering RLVR methods, token efficiency improvements, agent training infrastructure, and the shift from optimization-focused research to data-centric approaches in model development.

Key Questions Answered

• •Token Efficiency Over Time: GPT-5 to 5.1 maintained similar benchmark performance while dramatically reducing token consumption, enabling longer agent workflows and faster task completion. This metric matters more than wall-clock time for measuring model capability improvements.
• •RLVR Data Quality Spectrum: Post-training methods like RLHF and RLVR differ primarily in signal quality rather than optimization algorithms. Verifiable rewards from math problems provide cleaner training signals than human preference data, making data source selection more critical than gradient variance optimization.
• •Post-Training Infrastructure Complexity: Running RL training involves exponentially more moving parts than pre-training, with each task requiring different grading setups and external dependencies. This creates significantly more debugging surface area when monitoring production runs, especially during late-night troubleshooting sessions.
• •Skills Gap in ML Engineering: The industry lacks engineers proficient in both distributed systems and machine learning research. Frontier progress requires seamlessly switching between infrastructure bottlenecks and model improvements, but current education systems optimize for specialization rather than this dual expertise.