Skip to main content
Software Engineering Daily

Small AI Models with Yoeven Khemlani

40 min episode · 2 min read
·
Yoeven Khemlani

Episode

40 min

Read time

2 min

Topics

Design & UX, Artificial Intelligence, Software Development

AI-Generated Summary

Key Takeaways

  • Small model strategy: Train 70B parameter models instead of 400B by specializing for single use cases, enabling deployment on A100 GPUs rather than requiring H100s, reducing infrastructure costs while maintaining 97-98% accuracy for specific tasks like structured web scraping.
  • Prompt engine architecture: Routes prompts across five models simultaneously, uses mixture of agents technique where smaller models judge outputs, then converges on best answer. By tenth execution, system locks to single optimal model, ensuring consistency while initially guaranteeing quality through consensus.
  • GPU-poor methodology: Build all models to run on accessible hardware (A100, A10G) rather than premium GPUs, prioritizing deployability for enterprise self-hosting over raw performance. This distribution strategy enables customers to deploy on AWS, Azure, GCP, or on-premise infrastructure without restrictions.
  • Developer experience principle: Design SDK with complete TypeScript typing so developers never need documentation for basic usage. NPM install provides intuitive API structure modeled after Stripe, where method names and parameters are self-explanatory, reserving docs only for advanced configurations.

What It Covers

Yoeven Khemlani explains how Jigsawstack builds specialized small AI models (70B parameters) for backend automation tasks like web scraping, OCR, and translation, achieving 98% accuracy while remaining deployable and cost-efficient at $1.40 per million tokens.

Key Questions Answered

  • Small model strategy: Train 70B parameter models instead of 400B by specializing for single use cases, enabling deployment on A100 GPUs rather than requiring H100s, reducing infrastructure costs while maintaining 97-98% accuracy for specific tasks like structured web scraping.
  • Prompt engine architecture: Routes prompts across five models simultaneously, uses mixture of agents technique where smaller models judge outputs, then converges on best answer. By tenth execution, system locks to single optimal model, ensuring consistency while initially guaranteeing quality through consensus.
  • GPU-poor methodology: Build all models to run on accessible hardware (A100, A10G) rather than premium GPUs, prioritizing deployability for enterprise self-hosting over raw performance. This distribution strategy enables customers to deploy on AWS, Azure, GCP, or on-premise infrastructure without restrictions.
  • Developer experience principle: Design SDK with complete TypeScript typing so developers never need documentation for basic usage. NPM install provides intuitive API structure modeled after Stripe, where method names and parameters are self-explanatory, reserving docs only for advanced configurations.

Notable Moment

Jigsawstack benchmarked their OCR model against Mistral's self-proclaimed world's best OCR and found significant performance gaps, demonstrating that specialized small models from focused startups can outperform rushed releases from well-resourced companies attempting to enter adjacent markets.

Know someone who'd find this useful?

Episode Transcript

Jigsawstack is a startup that develops a suite of custom small models for tasks such as scraping, forecasting, VOCR, and translation. The platform is designed to support collaborative knowledge work, especially in research heavy or strategy driven environments. Yovan Kamlani is the founder of Jigsaw Stack, and he joins the podcast with Gregor Van to talk about making use of small models for diverse applications. Gregor Vann is a CTO and founder, currently working at the intersection of communication, security, and AI, and is based in Singapore. His latest venture, wintik.ai, reimagines what email can be in the AI era. For more on Gregor, find him at van.hk or on LinkedIn. Hi. Welcome to Software Engineering Daily. My guest today is Juergen Kemlani. Welcome, Juergen. Hey. Hey. Good to be here. So, Juergen, you are the founder of Jigsaw Stack, a ultimately, Singapore company, but you're now in The US, which is pretty well trodden path, I think, for this part of the world. So before we get into Jigsaw Stack, which, I cannot wait to get into, it's a very interesting product and very timely. I'd love to just hear a bit more about your background. I know this is, as I say, not your first rodeo when it comes to having founded a company. So, yeah, just tell us about your journey to Jigsawstack. Yeah. I think it didn't start very exciting or it's like every other engineer. So I'm an engineer like everyone else. I love building products. I love exploring new technologies and that's kind of like where I started. So I started as a game developer, which many choose not to take that path because it's one of the hardest industries to kind of break into from a revenue standpoint or for like a salary standpoint, like your initial career, but something I enjoyed did that for a bit. Love building, did love the industry, went into like the banking industry, didn't love the corporate industry, decided to leave that. And as I went to study my master's at Imperial, COVID hit right at the right moment. So I decided to drop out. I didn't want to pay with $200,000 just to sit in my dorm room. So I decided to drop out and eventually that's when I got inspired to start this company called Stay Our way back. I think it's like four years ago. It was a hotel aggregation company in Southeast Asia that basically did shop in bookings for, like, the pool, the gym, and different things within the hotel because domestic markets were growing at back then. Then that grew to a point. We went into co working spaces and then eventually we sold it. And I think that kind of kicked off my path of like, okay, startups is like where I want to be. I love building and I love building for people. Right. It's like, how can I make money from the things that I build? And that's …

Get the full transcript (8,741 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Software Engineering Daily transcripts →

You just read a 3-minute summary of a 37-minute episode.

Get Software Engineering Daily summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

Books, tools, and gear mentioned in this episode

SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.

Tools

  • by Mistral AI

    Jigsawstack benchmarked their OCR model against Mistral's self-proclaimed world's best OCR and found significant performance gaps, demonstrating that specialized small models from focused startups can outperform rushed releases from well-resourced companies attempting to enter adjacent markets.
  • by Stripe

    NPM install provides intuitive API structure modeled after Stripe, where method names and parameters are self-explanatory, reserving docs only for advanced configurations.

Gear

  • by NVIDIA

    Train 70B parameter models instead of 400B by specializing for single use cases, enabling deployment on A100 GPUs rather than requiring H100s, reducing infrastructure costs while maintaining 97-98% accuracy for specific tasks like structured web scraping.
  • by NVIDIA

    Train 70B parameter models instead of 400B by specializing for single use cases, enabling deployment on A100 GPUs rather than requiring H100s, reducing infrastructure costs while maintaining 97-98% accuracy for specific tasks like structured web scraping.
  • by NVIDIA

    Build all models to run on accessible hardware (A100, A10G) rather than premium GPUs, prioritizing deployability for enterprise self-hosting over raw performance.

company

  • Yoeven Khemlani explains how Jigsawstack builds specialized small AI models (70B parameters) for backend automation tasks like web scraping, OCR, and translation, achieving 98% accuracy while remaining deployable and cost-efficient at $1.40 per million tokens.

More from Software Engineering Daily

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Cybersecurity Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's AI & Machine Learning Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into Software Engineering Daily.

Every Monday, we deliver AI summaries of the latest episodes from Software Engineering Daily and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime