Software Verification in the Age of Coding Agents

Software testing is widely understood as essential to building reliable systems, and it's one of the hardest things to get right. A fundamental issue is that tests are good at confirming the behavior a team expects, but the most damaging failures tend to hide in the edge cases no one thought to check during development. Those blind spots are only growing as AI produces more code that no human has closely reviewed. Antithesis is a software verification company built around an approach called deterministic simulation testing. The platform runs a customer software in a fully controlled environment, exploring thousands of different paths through it at runtime. Crucially, the paths aren't prespecified, and they're not merely the expected cases. In this way, it can surface rare, damaging bugs that normal testing would never catch. Carl Spera is on the red team at Antithesis, and he also leads much of the company's AI strategy. In this episode, Carl joins Sean Falconer to discuss the Antithesis approach to software verification, why reliability tooling has been chronically underinvested, what makes stateful and distributed systems so hard to test, how deterministic simulation testing works under the hood, and much more. This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him. Carl, welcome to the show. Hey, Sean. Happy to be here. Yeah, absolutely. So we're going to get into all kinds of stuff today, but I thought just to start things off, could you give people maybe a sense of like what you actually do day to day? It seems like you're embedded with customers. You're tracking the shape of defects as they hit. What is it your day to day kind of look like? That's a great question. So I work at Antithesis and Antithesis is a software verification company. Right? And what my job is on the paper is I'm on a team called the red team. And this is not to be confused with red teams in... Like, a lot of times, teams are, like, more like a security team, maybe, like, an internal, like, red teaming security team. In this case, we named the red team after the idea that what if we hired a bunch of distributed systems experts, people who had spent their whole careers working on building distributed systems and testing them? What if we gave them essentially unlimited access to Intithesis to become experts in how to use Intithesis as a testing technology and then basically use that team to then go and help our customers use the technology to its fullest. So I would say my core goal is to become an expert in antithesis and then help our customers solve and find these very hard bugs in distributed systems. I also am taking a lead in a lot of our AI strategy. So I sort of bridge the gap between a customer facing role and more of like a product role. Okay. And …