Skip to main content
Eye on AI

#304 Matt Zeiler: Why Government And Enterprises Choose Clarifai For AI Ops

55 min episode · 2 min read
·
Matt Zeiler

Episode

55 min

Read time

2 min

Topics

Investing, Startups, Fundraising & VC

AI-Generated Summary

Key Takeaways

  • Inference optimization strategy: Clarifai achieves 65% lower time-to-first-token and 40% faster overall response times through CUDA kernel optimization, Python-to-C++ conversion, and speculative token prediction techniques that work across different accelerators without requiring specialized hardware.
  • Deployment flexibility advantage: The platform runs identically across air-gapped government networks, on-premise bare metal, customer VPCs, and multiple clouds (AWS, Azure, Google), allowing customers to start on-premise for cost savings then spill over to NeoCloud or hyperscalers as demand scales.
  • GPT-4o-mini performance economics: Running OpenAI's GPT-4o-mini on single GPUs delivers the optimal combination of intelligence, speed, and cost-effectiveness. This model enables competitive pricing while maintaining high throughput, making it superior to alternatives requiring eight GPUs for comparable intelligence levels.
  • Government AI adoption model: Intelligence analysts successfully train custom models independently using Clarifai's UI for labeling, template selection, and evaluation metrics without engineering support. This self-service capability proves essential for classified environments where external assistance faces restrictions.

What It Covers

Matt Zeiler, Clarifai CEO, discusses the company's evolution from computer vision pioneer to AI inference leader, detailing how software optimizations achieve 40% faster response times than competitors without specialized hardware.

Key Questions Answered

  • Inference optimization strategy: Clarifai achieves 65% lower time-to-first-token and 40% faster overall response times through CUDA kernel optimization, Python-to-C++ conversion, and speculative token prediction techniques that work across different accelerators without requiring specialized hardware.
  • Deployment flexibility advantage: The platform runs identically across air-gapped government networks, on-premise bare metal, customer VPCs, and multiple clouds (AWS, Azure, Google), allowing customers to start on-premise for cost savings then spill over to NeoCloud or hyperscalers as demand scales.
  • GPT-4o-mini performance economics: Running OpenAI's GPT-4o-mini on single GPUs delivers the optimal combination of intelligence, speed, and cost-effectiveness. This model enables competitive pricing while maintaining high throughput, making it superior to alternatives requiring eight GPUs for comparable intelligence levels.
  • Government AI adoption model: Intelligence analysts successfully train custom models independently using Clarifai's UI for labeling, template selection, and evaluation metrics without engineering support. This self-service capability proves essential for classified environments where external assistance faces restrictions.

Notable Moment

Zeiler recalls being among the first 20 people globally writing CUDA kernels for AI in 2011-2012, when adopting Alex Krizhevsky's shared kernels made his PhD experiments run 30 times faster overnight, transforming day-long waits into lunch-break turnarounds.

Know someone who'd find this useful?

Episode Transcript

It started with just Inference back in 2014. The the reason we started there, and having an API platform was inspirations like Stripe and Twilio, where they built a great developer product, and people just built amazing things on top of that. And they became kind of the engine for that. And we saw that there needs to be this engine for AI. And so that's the same product on the commercial side as public, sector side. We intentionally built a whole suite of UIs that make the system easy enough for anybody to use. And because of that, we've had even intelligence analysts train models themselves, doing all the labeling, picking a training template, looking at evaluation metrics of how they perform without our teams, being involved. That's key to making the API successful, which is ultimately what you're gonna use. Once you have a good model, you're gonna use the APIs to fire through lots and lots of data, for inference. But to get to that model, the UIs are really important. In business, they say you can have better, cheaper, or faster, but you only get to pick two. What if you could have all three at the same time? That's exactly what Coher, Thompson Reuters, and Specialized Bikes have. Since they upgraded to the next generation of the cloud, Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds. How is it faster? OCI's block storage gives you more operations per second. Cheaper? OCI costs up to 50% less for compute, 70% less for storage, and 80 less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is a cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com/ionai. Ionai all run together, eyeonai. That's oracle.com/ionai. I'm, Matt Zieler and founder and CEO of Clarify. And, my history in AI goes way back even into undergrad. That's kinda where it started. A little bit by luck, I was at University of Toronto and in a program where you have to decide if you wanna take, you know, many different options of engineering. And, I was deciding between the computer option and nanotechnology, and that's when I happened to run into one of Jeff Hinton's PhD students, a guy named Graham Taylor, and he happened to be my resident adviser on the floor I was living in. So that was the the luck of it all. And he showed me some of his research. And back then, he was generating videos that look realistic of a flame flickering, and he said it was all done by AI. And so fast forward to the day, everybody calls that generative AI. This was 2007. So we've been …

Get the full transcript (8,457 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Eye on AI transcripts →

You just read a 3-minute summary of a 52-minute episode.

Get Eye on AI summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

Books, tools, and gear mentioned in this episode

SignalCast may earn commission on purchases via these links.

Tools

  • by OpenAI

    Running OpenAI's GPT-4o-mini on single GPUs delivers the optimal combination of intelligence, speed, and cost-effectiveness. This model enables competitive pricing while maintaining high throughput, making it superior to alternatives requiring eight GPUs for comparable intelligence levels.
  • by Oracle

    SPONSORS: Oracle Cloud Infrastructure

company

  • ClarifaiBy guest
    Matt Zeiler, Clarifai CEO, discusses the company's evolution from computer vision pioneer to AI inference leader, detailing how software optimizations achieve 40% faster response times than competitors without specialized hardware.

More from Eye on AI

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's Investing & Markets Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into Eye on AI.

Every Monday, we deliver AI summaries of the latest episodes from Eye on AI and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime