AI Legal Software with Scott Stevenson

What It Covers

Scott Stevenson, co-founder of Spellbook, describes building an AI contract review and drafting tool for commercial lawyers. Starting as Rally in 2019, the company ran nearly 100 product experiments before launching its LLM-based copilot in 2022, now serving close to 2,000 paying law firms.

Key Questions Answered

• •Product-Market Fit Testing: Run experiments in three sequential stages before scaling: first test whether a landing page captures email signups, then whether users pay, then whether they return repeatedly. Most ideas fail at stage one. Retention is the only metric that cannot be faked by strong salespeople or generous contract terms.
• •AI-First vs. AI-Added: Launching a standalone AI product outperforms adding AI as a feature to an existing platform. Spellbook was rebuilt from scratch rather than appended to Rally, which prevented the new capability from being buried as item 20 of 20 in a feature list and allowed a fundamentally different user experience.
• •Hallucination Mitigation via Optionality: Present AI output as a menu of choices rather than a single authoritative answer. Spellbook surfaces roughly 100 suggested contract changes per review; lawyers accept approximately 50% of them. This removes the need for perfect accuracy and reframes the tool as a thread-finder rather than a decision-maker.
• •Custom Model Timing: Avoid investing heavily in fine-tuned or proprietary models prematurely. Spellbook's initial version was largely a GPT-3 wrapper with prompt engineering. Because foundation models improve rapidly, resources spent training narrow models can be rendered obsolete by the next public release, wasting capital that early-stage startups cannot afford.
• •Workflow-Embedded UX: Build AI tools directly inside the software users already operate rather than creating a separate chat interface. Spellbook runs inside Microsoft Word because lawyers refuse to leave it. Embedding suggestions within existing workflows increases adoption and preserves the user's sense of control, which is especially critical in high-stakes professional environments.