Exploit Development, Agentic Vulnerability Discovery, and Fake PoCs
Episode
45 min
Read time
2 min
Topics
Fundraising & VC, Artificial Intelligence, Software Development
AI-Generated Summary
Key Takeaways
- ✓Vulnerability vs. Exploit distinction: A vulnerability is the theoretical flaw; an exploit is the concrete implementation that proves attacker utility. Most AI models identify vulnerabilities well but consistently fail to bridge the gap to functional exploitation, meaning CVSS scores alone cannot drive patch prioritization without understanding real-world attacker utility for your specific environment.
- ✓POC quality tipping point: VulnerCheck's analysis of public proof-of-concept exploits across GitHub, GitLab, and CodeBerg found that in 2026, non-functional or low-quality POCs crossed 50% of all published submissions. Security teams should validate any public POC against a live target before treating it as confirmed attacker capability or using it to justify emergency remediation timelines.
- ✓AI exploit capability by vulnerability class: LLMs reliably produce functional exploits for open-source web application flaws — logic errors, token handling bugs, expression language injections, file leaks — but struggle significantly with closed-source firmware and proprietary x86 software requiring patch diffing or reverse engineering. Teams should allocate human researchers specifically to closed-source targets rather than relying on AI-generated output.
- ✓Go Exploit framework as LLM scaffold: VulnerCheck uses its open-source Go Exploit framework as a structured corpus when prompting LLMs for exploit development. Having a normalized, growing library of 50-plus new exploits per week reduces LLM hallucination and scaffolding errors. Security teams building AI-assisted research pipelines should establish a structured framework before prompting models, not after.
- ✓Guardrail workaround via local models: Proprietary American AI models have progressively tightened restrictions on offensive security tasks, with researchers reporting meaningful capability drops after Anthropic's Claude Opus 4.6/4.8. Anthropic's own benchmarks show GLM 5.3 running locally at roughly 90% of frontier model performance. Offensive security teams handling sensitive vulnerability data are migrating to self-hosted open-weight models to avoid both guardrails and data exposure risks.
What It Covers
Spencer Mc, Director of Exploit Development at VulnerCheck, covers how AI is reshaping vulnerability research and exploit development, the distinction between vulnerabilities and exploits, why over 50% of public proof-of-concept exploits on GitHub are now non-functional, and where agentic AI fits into offensive security workflows.
Key Questions Answered
- •Vulnerability vs. Exploit distinction: A vulnerability is the theoretical flaw; an exploit is the concrete implementation that proves attacker utility. Most AI models identify vulnerabilities well but consistently fail to bridge the gap to functional exploitation, meaning CVSS scores alone cannot drive patch prioritization without understanding real-world attacker utility for your specific environment.
- •POC quality tipping point: VulnerCheck's analysis of public proof-of-concept exploits across GitHub, GitLab, and CodeBerg found that in 2026, non-functional or low-quality POCs crossed 50% of all published submissions. Security teams should validate any public POC against a live target before treating it as confirmed attacker capability or using it to justify emergency remediation timelines.
- •AI exploit capability by vulnerability class: LLMs reliably produce functional exploits for open-source web application flaws — logic errors, token handling bugs, expression language injections, file leaks — but struggle significantly with closed-source firmware and proprietary x86 software requiring patch diffing or reverse engineering. Teams should allocate human researchers specifically to closed-source targets rather than relying on AI-generated output.
- •Go Exploit framework as LLM scaffold: VulnerCheck uses its open-source Go Exploit framework as a structured corpus when prompting LLMs for exploit development. Having a normalized, growing library of 50-plus new exploits per week reduces LLM hallucination and scaffolding errors. Security teams building AI-assisted research pipelines should establish a structured framework before prompting models, not after.
- •Guardrail workaround via local models: Proprietary American AI models have progressively tightened restrictions on offensive security tasks, with researchers reporting meaningful capability drops after Anthropic's Claude Opus 4.6/4.8. Anthropic's own benchmarks show GLM 5.3 running locally at roughly 90% of frontier model performance. Offensive security teams handling sensitive vulnerability data are migrating to self-hosted open-weight models to avoid both guardrails and data exposure risks.
Notable Moment
Spencer Mc described a case where someone used Claude autonomously to exploit a SQL injection vulnerability — not for malicious gain, but simply to jump a spin class waitlist. He flagged this as a signal that legal precedent around AI-assisted unauthorized access remains entirely unresolved.
Episode Transcript
Narrator: Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. Now onto the show. Daniel: Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my cohost, Chris Benson, who is a principal AI and autonomy research engineer. How are doing, Chris? Chris: Hey. Doing great. Going. Going. Going. How's it going? Daniel: Going. Going. Going. I mean, Chris, it seems like every day we're hearing about something like this, but today also was a was a was a doozy, I guess. You know, we... I I saw Wall Street Journal article saying, you know, AI discovered, exploit that affects 7070% of the web. Right? So this is happening, I guess, as we're recording this October 21. So we're hearing about this all the time, and, it conveniently happened that we had scheduled to, talk talk with us today on the podcast, the director of Exploit Development at Vulnerability, Spencer McIntyre. Welcome, Spencer. Spencer: Hi. Yeah. Thank you for having me. Daniel: Yeah. Definitely. Well, I I know, you know, today's today's events, we may not be able to, go into detail because we're still learning all the things, but it is evidence of something that is on the top of everyone's mind, which is how AI intersects with this world of cybersecurity, particularly around discovering exploits, developing exploits, the offensive defensive side. Excited to dig into some of that with you today. But our audience maybe isn't coming from not everyone's coming from a cybersecurity background. Could you help us understand, you know, a little bit about your background, how you fit into this world? People might find the director of exploit development title rather interesting. So just to help orient us kind of where do you fit within this world? What are you what are you doing right now with Vulnerability? That that sort of thing. Spencer: Yeah. Absolutely. I was really excited when they offered me the title of director of Exploit Development. Pretty awesome. I thought that was pretty cool. Chris: That's a pretty Spencer: cool I've always been really passionate about exploits. So that's kind of actually how I got into the information security industry. So my background started out in consulting, but being very interested in exploit development. At the time, I was under the impression, this was a long time ago, that you really needed to have a strong background in vulnerability …
Get the full transcript (7,873 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 42-minute episode.
Get Practical AI summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Practical AI
Education is Failing our Students in this AI-Driven World
Oct 15 · 43 min
Software Engineering Daily
Security in the Age of Instant Exploits
Oct 6
More from Practical AI
Narrative intelligence and the human advantage
Oct 8 · 44 min
Odd Lots
What the OpenAI-Hugging Face Hack Really Tells Us About AI Danger
Aug 17
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links.
Tools
- Go Exploit FrameworkRecommended
by VulnerCheck
“VulnerCheck uses its open-source Go Exploit framework as a structured corpus when prompting LLMs for exploit development. Having a normalized, growing library of 50-plus new exploits per week reduces LLM hallucination and scaffolding errors.”
by Anthropic
“Proprietary American AI models have progressively tightened restrictions on offensive security tasks, with researchers reporting meaningful capability drops after Anthropic's Claude Opus 4.6/4.8.”
- GLM 5.3Recommended
“Anthropic's own benchmarks show GLM 5.3 running locally at roughly 90% of frontier model performance. Offensive security teams handling sensitive vulnerability data are migrating to self-hosted open-weight models to avoid both guardrails and data exposure risks.”
More from Practical AI
We summarize every new episode. Want them in your inbox?
Education is Failing our Students in this AI-Driven World
Narrative intelligence and the human advantage
Open models and the future of Physical AI with NVIDIA
From AGENTS.md to Enterprise Deployment
How to get discovered in AI search
Similar Episodes
Related episodes from other podcasts
Software Engineering Daily
Oct 6
Security in the Age of Instant Exploits
Odd Lots
Aug 17
What the OpenAI-Hugging Face Hack Really Tells Us About AI Danger
Beyond Biotech
Jul 24
This nonprofit is building the ecosystem to cure epidermolysis bullosa
Cognitive Revolution
May 24
All Compute Is Food: Palisade's Jeffrey Ladish on AI Shutdown Resistance, Self-Replication & Ecology
The AI Breakdown
May 21
Anthropic Just Reset AI Expectations
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's AI & Machine Learning Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Practical AI.
Every Monday, we deliver AI summaries of the latest episodes from Practical AI and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime