Small AI Models with Yoeven Khemlani

What It Covers

Yoeven Khemlani explains how Jigsawstack builds specialized small AI models (70B parameters) for backend automation tasks like web scraping, OCR, and translation, achieving 98% accuracy while remaining deployable and cost-efficient at $1.40 per million tokens.

Key Questions Answered

• •Small model strategy: Train 70B parameter models instead of 400B by specializing for single use cases, enabling deployment on A100 GPUs rather than requiring H100s, reducing infrastructure costs while maintaining 97-98% accuracy for specific tasks like structured web scraping.
• •Prompt engine architecture: Routes prompts across five models simultaneously, uses mixture of agents technique where smaller models judge outputs, then converges on best answer. By tenth execution, system locks to single optimal model, ensuring consistency while initially guaranteeing quality through consensus.
• •GPU-poor methodology: Build all models to run on accessible hardware (A100, A10G) rather than premium GPUs, prioritizing deployability for enterprise self-hosting over raw performance. This distribution strategy enables customers to deploy on AWS, Azure, GCP, or on-premise infrastructure without restrictions.
• •Developer experience principle: Design SDK with complete TypeScript typing so developers never need documentation for basic usage. NPM install provides intuitive API structure modeled after Stripe, where method names and parameters are self-explanatory, reserving docs only for advanced configurations.