490: Large Language Misadventure

What It Covers

Aji Slater and Sally Hall examine their skepticism toward AI coding tools, discussing when large language models prove useful versus harmful. They explore quality limitations of AI-generated code, ethical concerns around training data, environmental impacts of data centers, and whether productivity gains justify societal costs including water usage and energy consumption.

Key Questions Answered

• •AI coding effectiveness: Large language models excel at linguistic tasks like text summarization and manipulation but produce average-quality code by design. Their probabilistic nature means they generate solutions based on what's statistically common rather than optimal, making it nearly impossible to create above-average code since the goal is predicting the most probable average response.
• •Code comprehension risk: Teams generating code they cannot fully understand create dangerous technical debt. When developers throw incomprehensible AI-generated code back into the LLM for modifications instead of understanding it themselves, systems drift further from human control. This accelerates creation of untouchable code sections that slow future development rather than improving long-term velocity.
• •Unethical training foundations: Models trained on copyrighted works without permission cannot be redeemed through future ethical practices. Like tainted evidence in legal proceedings, the foundational extraction of value against creators' permission permanently corrupts the tool. Companies have torrented books specifically because obtaining proper licensing would cost too much, prioritizing profit over legal and ethical obligations.
• •Environmental externalities: AI data centers disproportionately impact historically marginalized communities through water depletion and electricity price increases. Small towns with cheap land and utilities subsidize AI infrastructure while bearing the burden of resource scarcity. These placement decisions follow patterns of systemic inequality, putting facilities where resistance is weakest and land is cheapest.
• •Developer fulfillment trade-off: Solving problems through AI tools eliminates the satisfaction, learning, and growth that comes from crafting solutions. While capitalism prioritizes speed over experience, developers who find fulfillment in the journey of problem-solving rather than just having solutions may need to exit the industry if AI adoption becomes mandatory for employment.