Skip to main content
The Changelog

The world of open source metadata (Interview)

103 min episode · 2 min read
·
Andrew Nesbitt

Episode

103 min

Read time

2 min

Topics

Productivity, Relationships, Fundraising & VC

AI-Generated Summary

Key Takeaways

  • Critical Package Concentration: Only 0.01% of packages constitute 80% of all open source usage across ecosystems, translating to roughly 15,000 packages maintained by approximately one person each. This extreme asymmetry reveals how few individuals actually maintain the infrastructure powering modern software development globally.
  • Dependency Data Value: The 24.5 billion dependency relationships provide stronger usage signals than download counts or GitHub stars. When developers remove dependencies, it indicates real problems, unlike stars which persist indefinitely. This data enables tracking actual adoption patterns and identifying breaking changes affecting downstream users.
  • Package Manager Quirks: R package manager removes packages that fail to proactively fix compatibility issues with updated dependencies, creating reproducibility problems for scientific research. NPM contains roughly 1,000 case-sensitive package names despite being case-insensitive, and Maven's nested POM XML structures create parsing complexity across different historical formats.
  • Funding Gap Reality: Between 25-50% of critical packages have automated funding mechanisms like GitHub Sponsors or Open Collective, but individual sponsorships outnumber corporate contributions 10-to-1. Many GitHub Sponsors top earners sell digital goods rather than maintaining open source projects, distorting the sustainability model.
  • SBOM Enrichment Market: Organizations use Ecosystems to enrich software bills of materials with license information, security advisories, and maintainer data across multiple package managers. GitHub Actions drive weekday traffic spikes as CI pipelines automatically validate dependencies, demonstrating the shift toward automated supply chain security practices.

What It Covers

Andrew Nesbitt discusses Ecosystems, tracking 12 million packages across 35 ecosystems and 287 million repositories. The platform provides open source metadata for SBOM enrichment, security analysis, and research, processing 50 million daily API requests while maintaining sustainability through grants and licensing.

Key Questions Answered

  • Critical Package Concentration: Only 0.01% of packages constitute 80% of all open source usage across ecosystems, translating to roughly 15,000 packages maintained by approximately one person each. This extreme asymmetry reveals how few individuals actually maintain the infrastructure powering modern software development globally.
  • Dependency Data Value: The 24.5 billion dependency relationships provide stronger usage signals than download counts or GitHub stars. When developers remove dependencies, it indicates real problems, unlike stars which persist indefinitely. This data enables tracking actual adoption patterns and identifying breaking changes affecting downstream users.
  • Package Manager Quirks: R package manager removes packages that fail to proactively fix compatibility issues with updated dependencies, creating reproducibility problems for scientific research. NPM contains roughly 1,000 case-sensitive package names despite being case-insensitive, and Maven's nested POM XML structures create parsing complexity across different historical formats.
  • Funding Gap Reality: Between 25-50% of critical packages have automated funding mechanisms like GitHub Sponsors or Open Collective, but individual sponsorships outnumber corporate contributions 10-to-1. Many GitHub Sponsors top earners sell digital goods rather than maintaining open source projects, distorting the sustainability model.
  • SBOM Enrichment Market: Organizations use Ecosystems to enrich software bills of materials with license information, security advisories, and maintainer data across multiple package managers. GitHub Actions drive weekday traffic spikes as CI pipelines automatically validate dependencies, demonstrating the shift toward automated supply chain security practices.

Notable Moment

Nesbitt calculated hosting Ecosystems on AWS would cost 15 times more than dedicated bare metal servers in France and Amsterdam. By running individual Rails apps per service with separate Postgres databases, he maintains infrastructure affordability while processing billions of dependency relationships and serving 50 million daily API requests.

Know someone who'd find this useful?

Episode Transcript

Welcome friends. I'm Jared and you are listening to the change log where each week Adam and I interview the hackers, the leaders, and the innovators of the software world. We pick their brains, we learn from their failures, we get inspired by their accomplishments, and we have a whole lot of fun along the way. This episode features Andrew Nesbitt, who builds tools and open datasets to support, sustain, and secure critical digital infrastructure. He's been exploring the world of open source metadata for over a decade. First, with libraries.io and now with ecosystems, which tracks 12 plus million packages, 287,000,000 repos, 24,500,000,000 dependencies, and 1,900,000 maintainers. What has Andrew learned from all of this? Who is using this open dataset and how does he hope others can build on top top of it all? You're about to find out. But first, a big thank you to our partners at fly.io, the public cloud built for developers who ship. We love fly. We probably will too. Learn more at fly.io. Okay. Andrew Nesbitt talking ecosystems on the change log. Let's do it. Well, friends, agentic Postgres is here and from our friends over at Tiger Data. This is the very first database built for agents, and it's built to let you build faster. You know, a fun side note is 80% of Claude was built with AI. Over a year ago, 25% of Google's code was AI generated. It's safe to say that now it's probably close to a 100%. Most people I talk to, most developers I talk to right now, almost all their code is being generated. That's a different world. Here's the deal. Agents are the new developers. They don't click. They don't scroll. They call. They retrieve. They parallelize. They plug in your infrastructure to places you need it to perform, but your database is probably still thinking about humans only because that's kinda where Postgres is at. Tiger Data's philosophy is that when your agents need to spin up sandboxes, run migrations, query huge volumes of vector and text data, well, normal Postgres, it might choke, and so they fix that. Here's where we're at right now. Agentic Postgres delivers these three big leaps, native search and retrieval, instant zero copy forks, and MCP server plus your CLI plus a cool free tier. Now if this is intriguing at all, head over to tigerdata.com. Install the CLI, just three commands, spin up an Aginta Postgres service, and let your agents work at the speed they expect, not the speed of the old way. The new way, Aginta Postgres, it's built for agents, is designed to elevate your developer experience, and build the next big thing. Again, go to tigerdata.com to learn more. Today, we're joined by, for us, an old friend, but a long time no talk, Andrew Nesbitt is here with us. And, you know, Andrew, I came across ecosystems, which is eco. No. Ecosyste.ms domain. Nice domain hack. Hard to say out …

Get the full transcript (17,329 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all The Changelog transcripts →

You just read a 3-minute summary of a 100-minute episode.

Get The Changelog summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

Books, tools, and gear mentioned in this episode

SignalCast may earn commission on purchases via these links.

Tools

  • Sponsor: Tigris Data listed among episode sponsors.
  • Sponsor: Augment Code listed among episode sponsors.
  • Sponsor: Fly.io listed among episode sponsors.
  • EcosystemsBy guest
    Andrew Nesbitt discusses Ecosystems, tracking 12 million packages across 35 ecosystems and 287 million repositories. The platform provides open source metadata for SBOM enrichment, security analysis, and research, processing 50 million daily API requests.
  • by GitHub

    Between 25-50% of critical packages have automated funding mechanisms like GitHub Sponsors or Open Collective.
  • Between 25-50% of critical packages have automated funding mechanisms like GitHub Sponsors or Open Collective.
  • by GitHub

    GitHub Actions drive weekday traffic spikes as CI pipelines automatically validate dependencies, demonstrating the shift toward automated supply chain security practices.
  • Sponsor: Miro listed among episode sponsors.

More from The Changelog

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Cybersecurity Podcasts (2026) — ranked and reviewed with AI summaries.

You're clearly into The Changelog.

Every Monday, we deliver AI summaries of the latest episodes from The Changelog and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime