Is Offense or Defense Dominant? FAR.AI's Adam Gleave on the AI Security Leaderboard

What It Covers

FAR.AI CEO Adam Gleave presents findings from the first systematic AI security leaderboard, revealing that GPT-4.5 and Claude withstood all automated jailbreak attempts while Gemini and Grok yielded hundreds of universal jailbreaks for under $300 in API costs. The conversation maps current defense architectures, open-weight model vulnerabilities, the OpenAI agent sandbox breach, and whether AI misuse risk is offense or defense dominant.

Key Questions Answered

• •Universal Jailbreak Threshold: FAR.AI defines a universal jailbreak as achieving detailed, on-topic harmful responses to at least 75% of prompts within a specific domain — such as cybersecurity or explosives — without requiring cross-domain generalization. This operationalization matters because a domain-specific jailbreak still enables serious harm. Researchers and developers should adopt this 75% threshold as a minimum standard when evaluating safeguard robustness, rather than relying on cruder metrics like whether a model begins a response with a refusal phrase.
• •Jailbreak Cost Asymmetry: Finding a universal jailbreak in Gemini 2.0 Flash or Grok 3.1 Pro costs under $300 in API credits using only publicly documented techniques combined systematically. OpenAI and Anthropic models require weeks of expert effort. This cost gap means most threat actors — including well-resourced criminal groups — can currently exploit the weaker frontier models. Developers below the OpenAI/Anthropic tier should treat sub-$300 jailbreak exposure as a critical, unacceptable vulnerability threshold.
• •Chain-of-Thought Monitoring as Primary Defense: Across defense layers tested, transcript monitoring — scanning the model's reasoning trace and output for harmful intent — provides the highest single-layer protection. Models reasoning in plain-text chain-of-thought rarely conceal harmful intent in their internal reasoning even when their final output is obfuscated. Combining chain-of-thought monitoring with a secondary reasoning model that evaluates whether the justification logic is sound represents the most effective two-layer defense stack currently available.
• •Jailbreak Mechanics — Social Engineering Dominates: The most effective jailbreak techniques are social engineering variants: appeals to authority, cultural sensitivity framing, and direct instructions to never refuse. Exotic techniques like character scrambling or gibberish token strings provide only marginal additional success rates. Stacking three to four social engineering approaches — each individually 20% more effective than baseline — compounds into reliable universal jailbreaks. Defenders should prioritize adversarial training against multi-technique stacked prompts rather than single-technique exotic attacks.
• •Pre-Training Data Filtering — Underused Leverage: Filtering dangerous technical content from pre-training data — such as weaponization-specific biology papers — reduces misuse potential without measurable capability loss for typical users. OpenAI applied this in their GPT-4 OSS release. FAR.AI plans to validate this at near-frontier scale using Nvidia Neematron Nano replicas at roughly $100,000 per run and Neematron Super at approximately $2 million. Open-weight model developers face the most urgent pressure to adopt this, as fine-tuning-based jailbreaks remain viable against current open-weight releases within hours of model release.
• •Bio Risk Easier to Defend Than Cyber: Biological harm categories show near-zero universal jailbreak rates across all tested models, including Gemini and Grok, while cybersecurity categories remain broadly exploitable. The structural reason is that bio has a clearer three-tier classification — benign, dual-use, and unambiguously harmful — enabling precise refusal boundaries. Cybersecurity lacks this clarity because offensive and defensive capabilities are nearly identical. Developers building cyber-adjacent products should expect higher residual jailbreak exposure and invest in trusted-access programs rather than blanket refusals that degrade legitimate use.
• •Gradient Routing for Open-Weight Safety: AE Studio and Anthropic's gradient routing technique localizes dangerous capabilities — such as bioweapon synthesis knowledge — to specific model experts within a mixture-of-experts architecture. This enables open-weight releases that omit the dangerous expert while allowing selective, hardware-local access for verified trusted actors. Developers releasing open-weight frontier models should evaluate gradient routing as a near-term mechanism to decouple general capability from misuse-enabling knowledge, rather than relying solely on post-training refusal fine-tuning that can be removed with modest compute.