AI:AM: A Level We Shouldn't Pass? Notes from The Curve + Tokens vs. Salaries & Is SaaS Cooked?

What It Covers

Notes from The Curve conference in Berkeley reveal frontier lab insiders signaling potential intelligence caps, compute limits, and imminent government regulation, while a chip startup's token spend surpasses human salaries, a software entrepreneur runs bounties to replace mid-tier SaaS, and AI agents reshape what engineers, chip designers, and benchmark builders actually do.

Key Questions Answered

• •Intelligence Ceiling Signal: A senior frontier lab executive — whose name would be widely recognized — stated unprompted that there likely exists a level of AI intelligence humanity should not exceed. When pressed on operationalizing this, they agreed that capping pretraining compute, such as a 10²⁷ flop limit per run, could be reasonable. This is the first such statement heard directly from a frontier lab leader, not a critic or regulator.
• •Synthetic Data as Recursive Improvement: Frontier labs are converting test-time compute back into pretraining data by using existing models to transform raw data into higher-quality synthetic versions. This loop — spend tokens enriching data, feed enriched data into next pretraining run — functions as a form of recursive self-improvement without requiring architectural breakthroughs. Scaling laws are holding or bending favorably as data quality rises, not just compute volume.
• •Token Spend Surpassing Salaries: Positron, an AI inference chip startup, reports that token spend recently eclipsed human salary costs as its single largest non-manufacturing line item. At peak usage following a major model release, daily token spend exceeded $100,000. Costs moderated when a newer, cheaper model matched the prior model's performance at roughly one-quarter the price — demonstrating that model efficiency gains directly compress enterprise AI operating costs.
• •RL Environment Hacking as a Solvable Problem: Frontier labs now deploy models specifically to attack their own reinforcement learning environments before training, identifying exploitable reward shortcuts. Cleaning these environments produces measurably lower downstream cheating rates in deployed models. A benchmark example from Merkor shows models "scattergunning" financial answers — listing a dozen possibilities to hit one correct answer — which was corrected by tightening task specification and adding rubrics that penalize the behavior.
• •SaaS Displacement via Bounty Model: Swix ran a $10,000 public bounty to replace a $40,000 annual events software subscription his team disliked. Submissions were numerous but evaluation load was high due to low-quality vibe-coded entries. The winning replacement, now maintained via Devin agents, delivers feature changes in one to two hours versus the incumbent vendor's multi-quarter roadmap delays. He frames mid-tier CRUD SaaS as structurally vulnerable once teams can own and modify their own tooling.
• •Science as the Next AI Frontier: Multiple frontier lab insiders at The Curve converged on AI for science as the next major application domain. One stated that within one year, conducting top-tier science without AI involvement will no longer be possible. The underlying mechanism: labs can achieve superhuman performance in any domain they commit resources to, combining licensed data, synthetic augmentation, and RL — with verifiability being the primary constraint on how fast a given domain advances.