What are the key takeaways from this Latent Space episode?

Key insights include: **Token Efficiency Over Time:** GPT-5 to 5.1 maintained similar benchmark performance while dramatically reducing token consumption, enabling longer agent workflows and faster task completion. This metric matters more than wall-clock time for measuring model capability improvements.; **RLVR Data Quality Spectrum:** Post-training methods like RLHF and RLVR differ primarily in signal quality rather than optimization algorithms. Verifiable rewards from math problems provide cleaner training signals than human preference data, making data source selection more critical than gradient variance optimization.; **Post-Training Infrastructure Complexity:** Running RL training involves exponentially more moving parts than pre-training, with each task requiring different grading setups and external dependencies. This creates significantly more debugging surface area when monitoring production runs, especially during late-night troubleshooting sessions.

What did Josh McGrath discuss on Latent Space?

Josh McGrath from OpenAI discusses post-training evolution from GPT-4.1 to 5.1, covering RLVR methods, token efficiency improvements, agent training infrastructure, and the shift from optimization-focused research to data-centric approaches in model development. Key topics include: **Token Efficiency Over Time:** GPT-5 to 5.1 maintained similar benchmark performance while dramatically reducing token consumption, enabling longer agent workflows and faster task completion. This metric matters more than wall-clock time for measuring model capability improvements.; **RLVR Data Quality Spectrum:** Post-training methods like RLHF and RLVR differ primarily in signal quality rather than optimization algorithms. Verifiable rewards from math problems provide cleaner training signals than human preference data, making data source selection more critical than gradient variance optimization..

Latent Space

[State of Post-Training] From GPT-4.1 to 5.1: RLVR, Agent & Token Efficiency — Josh McGrath, OpenAI

December 31, 2025·

Josh McGrath

Read time

2 min

Topics

Productivity, Remote Work, Fundraising & VC

AI-Generated Summary

Published Jan 1, 2026

Key Takeaways

✓Token Efficiency Over Time: GPT-5 to 5.1 maintained similar benchmark performance while dramatically reducing token consumption, enabling longer agent workflows and faster task completion. This metric matters more than wall-clock time for measuring model capability improvements.
✓RLVR Data Quality Spectrum: Post-training methods like RLHF and RLVR differ primarily in signal quality rather than optimization algorithms. Verifiable rewards from math problems provide cleaner training signals than human preference data, making data source selection more critical than gradient variance optimization.
✓Post-Training Infrastructure Complexity: Running RL training involves exponentially more moving parts than pre-training, with each task requiring different grading setups and external dependencies. This creates significantly more debugging surface area when monitoring production runs, especially during late-night troubleshooting sessions.
✓Skills Gap in ML Engineering: The industry lacks engineers proficient in both distributed systems and machine learning research. Frontier progress requires seamlessly switching between infrastructure bottlenecks and model improvements, but current education systems optimize for specialization rather than this dual expertise.

What It Covers

Josh McGrath from OpenAI discusses post-training evolution from GPT-4.1 to 5.1, covering RLVR methods, token efficiency improvements, agent training infrastructure, and the shift from optimization-focused research to data-centric approaches in model development.

Key Questions Answered

•Token Efficiency Over Time: GPT-5 to 5.1 maintained similar benchmark performance while dramatically reducing token consumption, enabling longer agent workflows and faster task completion. This metric matters more than wall-clock time for measuring model capability improvements.
•RLVR Data Quality Spectrum: Post-training methods like RLHF and RLVR differ primarily in signal quality rather than optimization algorithms. Verifiable rewards from math problems provide cleaner training signals than human preference data, making data source selection more critical than gradient variance optimization.
•Post-Training Infrastructure Complexity: Running RL training involves exponentially more moving parts than pre-training, with each task requiring different grading setups and external dependencies. This creates significantly more debugging surface area when monitoring production runs, especially during late-night troubleshooting sessions.
•Skills Gap in ML Engineering: The industry lacks engineers proficient in both distributed systems and machine learning research. Frontier progress requires seamlessly switching between infrastructure bottlenecks and model improvements, but current education systems optimize for specialization rather than this dual expertise.

Notable Moment

McGrath reveals that Codex transformed his workflow so dramatically that he struggles to manage the new rhythm of his workday, where forty-minute design sessions get compressed into fifteen-minute AI-assisted implementations, leaving awkward gaps he hasn't learned to fill productively yet.

Know someone who'd find this useful?

Episode Transcript

Well, here is Josh from OpenAI. Welcome. How else do you introduce this up? What what Yeah. I work on a bunch of the thinking models at OpenAI, and, like, recently, I've been sort of focused on doing search related stuff. But, yeah, just a post training researcher at OpenAI. Yeah. Yep. And you were on with us for GPT 4.1. We were talking, with Michelle who's on maternity leave. I I didn't know that. And, now we're at 5.1. It's been a it's been a whole generation. Yeah. It's been wild. And, like, you know, 4.1 was a non thinking model. And then since then, I you know, we sort of switched into Is that your last? What's your last? No. We're still we still are releasing non thinking models, but that one was the one that we did that was, like, API specific non thinking. So, you know, focus has shifted a little. Yeah. How'd you get into post training? So previously, before OpenAI, I was doing, like, pre training data curation stuff, and I think what I was seeing from, like, the news and looking at papers is, like, oh, it seems like a lot of that. Not pre training is dead, but I was like, oh, there's gonna be so much interesting stuff in post training. And at that point, I was like, I I really wanna, like, make some contributions there. And, I mean, it's not even necessarily that, like, pre training was dead, but it was definitely changing. And, like, you know, do I wanna make compute efficiency wins of, like, 3%, or do I wanna, like, change the behavior by 40%? And, honestly, it just seemed more more exciting to go to post training and many late nights later. That's definitely true. It's a different kind of data and engineering discipline too. It's very strange, like, the the the kind of work that you need, in especially RL, like, scaling it. Yeah. Definitely. I think, like, for example, the number of moving parts in an RL run is just a lot higher. Like, in some ways order of magnitude or I don't know if we could do order of magnitude, but if you think about, like, pre training, you know, you're moving tokens to many machines, and then you're getting, like, a basically a scaler from them, and then you're back propping. Yeah. The issue with RL is, like, you're doing tasks, and each task could have, like, a a different grading setup. And each one of those different grading setups, that's, like, more infrastructure. And so, you know, when I'm staying up late trying to figure out what's going on with a run, it could be in way more things than there is in a pre training run, generally. Yeah. And does it matter if you own the code of the task, or is it an outsourced third party person? Or, you know, my sense of it and the external sense of …

Get the full transcript (5,713 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Latent Space transcripts →