Tools mentioned by Sean Macgregor

“Sean MacGregor, founder of the AI Incident Database and cofounder of the AI Verification and Evaluation Research Institute, explains how AI safety incidents are documented. The database contains over 5,000 human-annotated reports across 1,000+ discrete incidents.”

“Benchmarks like BBQ for bias testing operate within specific prompt distributions that may not generalize to actual deployment environments.”

“The BenchRisk meta-evaluation project found many benchmarks lack sufficient documentation and evidence, essentially providing trust-me-bro level receipts rather than rigorous validation for real-world safety claims.”