Skip to main content
Decoder

The surprising case for AI judges

73 min episode · 3 min read
·
Bridget Mccormack

Episode

73 min

Read time

3 min

Topics

Leadership, Artificial Intelligence, Software Development

AI-Generated Summary

Key Takeaways

  • Justice System Accessibility Crisis: 92% of Americans cannot afford legal help for their problems, including most small and medium businesses. State courts adjudicate 3-4 million cases annually in Michigan alone, with the majority of parties unrepresented. This creates a fundamental access problem where the modal case type in state courts is consumer debt, with most cases dismissed due to procedural failures rather than merits.
  • AI Arbitrator Architecture: The system employs approximately 20 specialized agents operating across the arbitration process. Front-end agents parse claims, organize arguments, identify legal elements, and verify understanding with parties through iterative feedback loops. Reasoning agents then analyze the dispute summary, while a human arbitrator remains in the loop throughout, ultimately issuing the final award after reviewing and modifying the AI-generated draft as needed.
  • Procedural Fairness Through Transparency: The system addresses trust by ensuring parties feel heard through explicit confirmation loops. Agents present their understanding of claims, elements, evidence, and legal frameworks, then ask parties for verification. This iterative process continues until both parties confirm accurate understanding, creating transparency that human judges rarely provide. Research shows parties grow trust in institutions when they understand the process and reasoning, even when losing.
  • Error Rates in Human Judgment: Appellate courts reverse lower court decisions at significant rates, indicating systemic human error. DNA exoneration data reveals wrongful conviction rates of 3-5% in criminal cases, demonstrating judges and juries make substantial mistakes. McCormack argues this error rate is acceptable for shooting free throws but unacceptable for landing planes, and the justice system should operate more like the latter with AI assistance.
  • Governed AI Systems Required: Generic large language models like ChatGPT would hallucinate if used directly for arbitration. The AAA system requires governance through training on specific dispute types, grounding in relevant legal frameworks, and continuous human oversight. The platform launched narrowly with documents-only construction cases because the team could build a properly governed agentic system using their historical case library and construction arbitrator collaboration for that specific domain.

What It Covers

Bridget McCormack, former Michigan Supreme Court Chief Justice and current American Arbitration Association CEO, discusses the AI arbitrator system launched in November 2024. The platform uses multiple AI agents to resolve documents-only construction disputes, with one active case. McCormack argues AI can increase access to justice while addressing concerns about hallucinations, bias, and trust in automated legal systems.

Key Questions Answered

  • Justice System Accessibility Crisis: 92% of Americans cannot afford legal help for their problems, including most small and medium businesses. State courts adjudicate 3-4 million cases annually in Michigan alone, with the majority of parties unrepresented. This creates a fundamental access problem where the modal case type in state courts is consumer debt, with most cases dismissed due to procedural failures rather than merits.
  • AI Arbitrator Architecture: The system employs approximately 20 specialized agents operating across the arbitration process. Front-end agents parse claims, organize arguments, identify legal elements, and verify understanding with parties through iterative feedback loops. Reasoning agents then analyze the dispute summary, while a human arbitrator remains in the loop throughout, ultimately issuing the final award after reviewing and modifying the AI-generated draft as needed.
  • Procedural Fairness Through Transparency: The system addresses trust by ensuring parties feel heard through explicit confirmation loops. Agents present their understanding of claims, elements, evidence, and legal frameworks, then ask parties for verification. This iterative process continues until both parties confirm accurate understanding, creating transparency that human judges rarely provide. Research shows parties grow trust in institutions when they understand the process and reasoning, even when losing.
  • Error Rates in Human Judgment: Appellate courts reverse lower court decisions at significant rates, indicating systemic human error. DNA exoneration data reveals wrongful conviction rates of 3-5% in criminal cases, demonstrating judges and juries make substantial mistakes. McCormack argues this error rate is acceptable for shooting free throws but unacceptable for landing planes, and the justice system should operate more like the latter with AI assistance.
  • Governed AI Systems Required: Generic large language models like ChatGPT would hallucinate if used directly for arbitration. The AAA system requires governance through training on specific dispute types, grounding in relevant legal frameworks, and continuous human oversight. The platform launched narrowly with documents-only construction cases because the team could build a properly governed agentic system using their historical case library and construction arbitrator collaboration for that specific domain.
  • B2B Agentic Commerce Implications: Walmart already uses agents to negotiate and execute contracts, with estimates suggesting 40% of B2B contracts may be agent-negotiated by 2027. This creates demand for automated dispute resolution when agents make mistakes. McCormack positions the AI arbitrator as necessary infrastructure for agentic commerce, arguing the system is only as reliable as the process for fixing breakdowns in automated contract execution.

Notable Moment

McCormack recounts confronting a Michigan probate judge who publicly criticized her on a statewide listserv for statements made at a forum. When she called to explain she had not attended that event, providing an alibi, the judge simply insisted she had been there. His refusal to acknowledge factual reality demonstrates how human judges can be unreliable in ways that properly audited AI systems cannot.

Know someone who'd find this useful?

Episode Transcript

Support for Decoder comes from Adobe. Life is unpredictable, and that means you need your projects to adapt with whatever gets thrown at you. That means mastering the ability to pivot and collaborate with others to reach your goals. Adobe gets that, which is why they made a tool that's just as flexible as you are, PDF Spaces and Acrobat Studio. Your PDF files are no longer static. Instead, they're living documents that flex with you and your project's needs. Learn more at adobe.com slash do that with Acrobat. It's true that some things change as we get older. But if you're a woman over 40 and you're dealing with insomnia, brain fog, moodiness, and weight gain, you don't have to accept it as just another part of aging. And with Mitty Health, you can get help and stop pushing through it alone. The experts at Mitty understand that all these symptoms can be connected to the hormonal changes that happen around menopause, and Middi can help you feel more like yourself again. Many healthcare providers aren't trained to treat or even recognize menopause symptoms. Middi clinicians are menopause experts. They're dedicated to providing safe, effective, FDA approved solutions for dozens of hormonal symptoms, not just hot flashes. Most importantly, they're covered by insurance. Ninety one percent of MiddI patients get relief from symptoms within just two months. You deserve to feel great. Book your virtual visit today at joinmidi.com. That's join,midi,.com. Hi. This is Bella Freud. I'm the host of Fashion Neurosis. This week on the show, Esther Perel is on my couch. Erotic recovery is part of trauma healing. God. That's interesting. It's not the reward at the end. Yeah. That's the difference. And I think we both, come together around that construct. Yeah. Find Fashion Neurosis on YouTube or wherever you get your podcasts. Hello and welcome to Decoder. I'm Eli Patel, editor in chief of The Verge, and Decoder is my show about big ideas and other problems. Today, we're gonna talk about the role AI might play in deciding legal disputes. Not just doing research and drafting memos, actually deciding who's right and who's wrong, and who should pay. My guest today is Bridget McCormack, the former chief justice of the Michigan Supreme Court, and now the president and CEO of the American Arbitration Association. You've probably heard of arbitration before. It's a form of dispute resolution that allows two parties to resolve conflicts outside of the formal court system using a third neutral party, the arbitrator, settlement. You may have never found yourself in arbitration, but you've almost certainly signed an arbitration clause in one of the many contracts in terms of service agreements that all of us have to sign all the time. Arbitration can be much faster, cheaper, and easier than going to court. So it's become a favored way of resolving disputes between businesses. It's also, as it turns out, many employers and large corporations defend against lawsuits, because they can …

Get the full transcript (14,739 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Decoder transcripts →

You just read a 3-minute summary of a 70-minute episode.

Get Decoder summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

More from Decoder

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Tech Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's AI & Machine Learning Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into Decoder.

Every Monday, we deliver AI summaries of the latest episodes from Decoder and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime