What are the key takeaways from this Hidden Forces episode?

Key insights include: **Inference-Time Scaling:** Models now spend more compute during the answer phase rather than training, using chain-of-thought reasoning to explore multiple solution paths before responding. This approach yields better performance on math, coding, and scientific tasks without requiring larger model sizes.; **Prompt Engineering Impact:** Users who provide detailed context, persona descriptions, and scaffolding get significantly better responses because they help navigate the model's high-dimensional answer space. Poor prompting accounts for much response variability, not just model limitations, giving informed users a measurable advantage.; **Model Regression Trade-offs:** ChatGPT-4o outperforms GPT-5 at writing tasks because foundation models involve hundreds of competing optimization signals across domains. Each update pulls the model in different directions, creating inevitable capability regressions in some areas while improving others, making consistent performance impossible.

What did Nathan Benaich discuss on Hidden Forces?

Nathan Benaich, founder of Air Street Capital and creator of the annual State of AI Report, examines breakthrough developments in artificial intelligence, including DeepSeek's innovations, reasoning models, and the shift from pre-training to inference-time scaling. Key topics include: **Inference-Time Scaling:** Models now spend more compute during the answer phase rather than training, using chain-of-thought reasoning to explore multiple solution paths before responding. This approach yields better performance on math, coding, and scientific tasks without requiring larger model sizes.; **Prompt Engineering Impact:** Users who provide detailed context, persona descriptions, and scaffolding get significantly better responses because they help navigate the model's high-dimensional answer space. Poor prompting accounts for much response variability, not just model limitations, giving informed users a measurable advantage..

How long is this episode of Hidden Forces?

This episode is 53 minutes long. SignalCast provides an AI-generated summary so you can get the key insights in about 3 minutes.

Hidden Forces

Investing on the Front Lines of the AI Arms Race | Nathan Benaich

November 10, 2025

53 min episode · 2 min read

Nathan Benaich

Episode

53 min

Read time

2 min

Topics

Investing, Startups, Artificial Intelligence

AI-Generated Summary

Published Dec 22, 2025

Key Takeaways

✓Inference-Time Scaling: Models now spend more compute during the answer phase rather than training, using chain-of-thought reasoning to explore multiple solution paths before responding. This approach yields better performance on math, coding, and scientific tasks without requiring larger model sizes.
✓Prompt Engineering Impact: Users who provide detailed context, persona descriptions, and scaffolding get significantly better responses because they help navigate the model's high-dimensional answer space. Poor prompting accounts for much response variability, not just model limitations, giving informed users a measurable advantage.
✓Model Regression Trade-offs: ChatGPT-4o outperforms GPT-5 at writing tasks because foundation models involve hundreds of competing optimization signals across domains. Each update pulls the model in different directions, creating inevitable capability regressions in some areas while improving others, making consistent performance impossible.
✓DeepSeek Cost Narrative: The reported five million dollar training cost for DeepSeek R1 only covered the final qualifying run, excluding all research and development, data annotation, infrastructure, and prior experimental training runs. This mirrors reporting only a Formula One qualifying lap cost while ignoring the entire race weekend expenses.

What It Covers

Nathan Benaich, founder of Air Street Capital and creator of the annual State of AI Report, examines breakthrough developments in artificial intelligence, including DeepSeek's innovations, reasoning models, and the shift from pre-training to inference-time scaling.

Key Questions Answered

•Inference-Time Scaling: Models now spend more compute during the answer phase rather than training, using chain-of-thought reasoning to explore multiple solution paths before responding. This approach yields better performance on math, coding, and scientific tasks without requiring larger model sizes.
•Prompt Engineering Impact: Users who provide detailed context, persona descriptions, and scaffolding get significantly better responses because they help navigate the model's high-dimensional answer space. Poor prompting accounts for much response variability, not just model limitations, giving informed users a measurable advantage.
•Model Regression Trade-offs: ChatGPT-4o outperforms GPT-5 at writing tasks because foundation models involve hundreds of competing optimization signals across domains. Each update pulls the model in different directions, creating inevitable capability regressions in some areas while improving others, making consistent performance impossible.
•DeepSeek Cost Narrative: The reported five million dollar training cost for DeepSeek R1 only covered the final qualifying run, excluding all research and development, data annotation, infrastructure, and prior experimental training runs. This mirrors reporting only a Formula One qualifying lap cost while ignoring the entire race weekend expenses.

Notable Moment

Benaich reveals that telling ChatGPT to think step by step two years ago improved performance because it decomposed complex tasks into smaller hops, allowing the system to debug its reasoning. This observation directly led developers to train models with explicit reasoning traces from domain experts.

Know someone who'd find this useful?

Episode Transcript

What's up, everybody? My name is Demetri Kofinas, and you're listening to Hidden Forces, a podcast that inspires investors, entrepreneurs, and everyday citizens to challenge consensus narratives and learn how to think critically about the systems of power shaping our world. My guest in this episode of Hidden Forces is Nathan Benesh, founder and general partner of Airstream Capital and the creator of the annual State of AI Report, an open access compendium that attracts advances across AI research, industry, policy, and geopolitics. Nathan and I spend the first hour of our conversation today exploring some of the most important AI breakthroughs of the year. We unpack the deep seek moment, dig into some of the advancements made by the latest reasoning models, and why there appears to be a regression in capabilities across certain domains and artificial intelligence at the same time as we are seeing marked improvements in reasoning heavy use cases like coding and scientific research. The second hour turns to a conversation about the commercial implications and geopolitical dynamics of the AI arms race, including China's strategy to become the leader in open weight models, tooling, the industries, sectors, and professions most ripe for disruption, where the investment opportunities are, whether we're in a bubble comparable to the nineteen nineties Internet boom, and how export controls, energy constraints, and regulatory red tape could play an outsized role in shaping the trajectory of the current arms race. Lastly, we look at where along the AI stack most of the value is likely to accrue, from the underlying picks and shovels, through the foundation models, to the apps that ride on top of them, and what all this means for labor markets, education, and the cadence of scientific discovery. If you want access to all of this conversation, go to hiddenforces.io/subscribe and join our premium feed, which you can listen to on your mobile device using your favorite podcast app just like you're listening to this episode right now. If you wanna join in on the conversation and become a member of the Hidden Forces Genius community, which includes q and a calls with guests, discounted access to third party research and analysis, and in person events like our intimate dinners and weekend retreats. You can also do that on our subscriber page. And if you still have questions, feel free to send an email to info@hiddenforces.io, and I or someone from our team will get right back to you. And with that, please enjoy this deeply informative and timely conversation with my guest, Nathan Benesh. Nathan Benesh, welcome to Hidden Forces. Thanks for having me. It's my pleasure to have you on, Nathan. I'm actually very excited to have you on the podcast. One of our genius members, Panos Papadopoulos, introduced us. He's a VC based out of Athens who we also need to get on the podcast at some point. So before we get into the substance of your latest annual report, which is, I should …

Get the full transcript (9,741 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Hidden Forces transcripts →