315 | Branden Fitelson on the Logic and Use of Probability
Episode
88 min
Read time
2 min
Topics
Software Development, Psychology & Behavior, Philosophy & Wisdom
AI-Generated Summary
Key Takeaways
- ✓Two-Dimensional Argument Strength: Strong arguments require both high probability (conclusion likely given premises) and high relevance (evidence actually changes probability). Diagnostic tests demonstrate this: a pregnancy test for a biologically male person shows high reliability but zero relevance, illustrating why both dimensions matter for rational belief updating.
- ✓Bayes Factors Over Posteriors: Scientific papers report likelihood ratios (true positive rate divided by false positive rate) rather than posterior probabilities because researchers cannot know readers' prior beliefs. This base factor represents objective, invariant information discoverable in laboratories, while posterior probabilities depend on individual background knowledge and assumptions.
- ✓Confirmation Versus Probability: Evidence can strongly confirm a hypothesis while leaving it improbable, or weakly confirm while making it highly probable. The base rate fallacy occurs when people confuse these dimensions: rare disease with reliable test yields low probability despite high confirmation, causing systematic reasoning errors in medical and scientific contexts.
- ✓Falsification Power: Popper's insight about falsification has quantitative validity: seeking counterexamples provides more confirmational power than seeking positive instances. In the Wason selection task, checking the seven card (potential falsifier) is more informative than checking the three card (potential confirmer), though people systematically reverse this ordering due to confirmation bias.
- ✓Pluralist Bayesian Framework: No single probability function works for all arguments across contexts. Each scientific domain requires constructing appropriate probability models with context-specific assumptions and idealizations. Particle physics generates such powerful likelihood ratios that prior probabilities barely matter, while other sciences remain highly sensitive to priors, requiring explicit model construction for each case.
What It Covers
Philosopher Branden Fitelson explains how probability theory applies to scientific reasoning, distinguishing between objective physical probabilities and epistemic probabilities used to evaluate evidence strength, confirmation, and argument quality across different scientific contexts.
Key Questions Answered
- •Two-Dimensional Argument Strength: Strong arguments require both high probability (conclusion likely given premises) and high relevance (evidence actually changes probability). Diagnostic tests demonstrate this: a pregnancy test for a biologically male person shows high reliability but zero relevance, illustrating why both dimensions matter for rational belief updating.
- •Bayes Factors Over Posteriors: Scientific papers report likelihood ratios (true positive rate divided by false positive rate) rather than posterior probabilities because researchers cannot know readers' prior beliefs. This base factor represents objective, invariant information discoverable in laboratories, while posterior probabilities depend on individual background knowledge and assumptions.
- •Confirmation Versus Probability: Evidence can strongly confirm a hypothesis while leaving it improbable, or weakly confirm while making it highly probable. The base rate fallacy occurs when people confuse these dimensions: rare disease with reliable test yields low probability despite high confirmation, causing systematic reasoning errors in medical and scientific contexts.
- •Falsification Power: Popper's insight about falsification has quantitative validity: seeking counterexamples provides more confirmational power than seeking positive instances. In the Wason selection task, checking the seven card (potential falsifier) is more informative than checking the three card (potential confirmer), though people systematically reverse this ordering due to confirmation bias.
- •Pluralist Bayesian Framework: No single probability function works for all arguments across contexts. Each scientific domain requires constructing appropriate probability models with context-specific assumptions and idealizations. Particle physics generates such powerful likelihood ratios that prior probabilities barely matter, while other sciences remain highly sensitive to priors, requiring explicit model construction for each case.
Notable Moment
Fitelson reveals that Kahneman and Tversky's own research papers commit the same reasoning pattern they criticize in subjects: reporting base factors and likelihood ratios rather than posterior probabilities, implicitly acknowledging that scientists cannot determine how probable hypotheses are without knowing readers' prior beliefs and background knowledge.
Episode Transcript
Hello, everyone, and welcome to the Mindscape podcast. I'm your host, Sean Carroll. One of the things that I always like to say about science and how it gets done is that science never proves things. This is something that is an important feature of science, especially in the modern world where what science does, how it reaches conclusions, how trustworthy it is, these are all under contestation by different parts of society. So it's important to understand what science is and how it actually reaches its conclusions. And the the claim that science never proves things, which is something that most scientists would go along with me on, comes from a comparison to real proof in mathematics or for that matter in logic. You know, most scientists have taken some math classes at least enough to know what it means to prove something in the old fashioned sense of Euclid and geometry or Aristotle and logic proving a conclusion from some well articulated premises. In the philosophical study of logic, this is known as deductive reasoning. You have some premises and you reach a conclusion. And science just doesn't go that way. Right? Science looks at the world. It looks at all sorts of things in the world, and it tries to figure out what the patterns are that the world follows. Always knowing that tomorrow, you might do a new experiment that will overturn your best guess as to what the pattern was, or maybe someone will do something as simple as just thinking of a better pattern. Right? A theoretical physicist coming up with a better idea for what the laws of physics really are. So if science doesn't prove things, if it just sort of comes closer and closer in some sense to getting it right, then what is what's going on? You know, one very common idea about what's going on is inductive logic rather than deductive logic. In inductive logic, we begin to see a pattern. You know, a, b, c, d, e, f, g. The next one is probably gonna be h. Right? Because we think that probably you're just mentioning the alphabet in alphabetical order. But there's all sorts of paradoxes that come up when you do inductive logic, like, how do you know that it's not a, b, c, d, e, f, z? That's a that's a sequence of letters that you that you could have. My old math teacher in college used to hate those SAT questions or standardized test questions that would give you a series of numbers and ask you to guess the next one. Because he said, I can I can make any number I want? I can come with a formula that would give you any number I want after the ones that you already showed me. So philosophers, unsurprisingly, are very interested in making as rigorous and careful as possible this idea of either induction or whatever should replace induction as the logic of understanding …
Get the full transcript (16,555 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 85-minute episode.
Get Sean Carroll's Mindscape summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Sean Carroll's Mindscape
367 | Jared Diamond on the Course of History and the Role of leaders
Sep 7 · 77 min
In Our Time
Condorcet
Feb 8
More from Sean Carroll's Mindscape
366 | Jim Al-Khalili on Time, Quantum, Biology, and Cosmology
Aug 31 · 75 min
Software Engineering Daily
A Rust Framework to Simplify Distributed Systems
Sep 10
More from Sean Carroll's Mindscape
We summarize every new episode. Want them in your inbox?
367 | Jared Diamond on the Course of History and the Role of leaders
366 | Jim Al-Khalili on Time, Quantum, Biology, and Cosmology
365 | Vitor Cardoso on Why Black Holes Are Special
364 | Stuart Firestein on How Science Relies on Ignorance and Failure
363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins
Similar Episodes
Related episodes from other podcasts
In Our Time
Feb 8
Condorcet
Software Engineering Daily
Sep 10
A Rust Framework to Simplify Distributed Systems
Modern Wisdom
Sep 5
Couples Therapist: “The One Rule Every Relationship Must Live By” - Stan Tatkin -#1146
Stuff You Should Know
Sep 3
Social Identity Theory: Your Group Rules, Others Drool
All-In with Chamath, Jason, Sacks & Friedberg
Aug 26
Eric Weinstein: The State of American Science, Breakthrough Coverups, and the Danger of Physics
Explore Related Topics
This podcast is featured in Best Science Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Software Engineering Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Sean Carroll's Mindscape.
Every Monday, we deliver AI summaries of the latest episodes from Sean Carroll's Mindscape and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime