Skip to main content
Sean Carroll's Mindscape

315 | Branden Fitelson on the Logic and Use of Probability

88 min episode · 2 min read
·
Branden Fitelson

Episode

88 min

Read time

2 min

Topics

Software Development, Psychology & Behavior, Philosophy & Wisdom

AI-Generated Summary

Key Takeaways

  • Two-Dimensional Argument Strength: Strong arguments require both high probability (conclusion likely given premises) and high relevance (evidence actually changes probability). Diagnostic tests demonstrate this: a pregnancy test for a biologically male person shows high reliability but zero relevance, illustrating why both dimensions matter for rational belief updating.
  • Bayes Factors Over Posteriors: Scientific papers report likelihood ratios (true positive rate divided by false positive rate) rather than posterior probabilities because researchers cannot know readers' prior beliefs. This base factor represents objective, invariant information discoverable in laboratories, while posterior probabilities depend on individual background knowledge and assumptions.
  • Confirmation Versus Probability: Evidence can strongly confirm a hypothesis while leaving it improbable, or weakly confirm while making it highly probable. The base rate fallacy occurs when people confuse these dimensions: rare disease with reliable test yields low probability despite high confirmation, causing systematic reasoning errors in medical and scientific contexts.
  • Falsification Power: Popper's insight about falsification has quantitative validity: seeking counterexamples provides more confirmational power than seeking positive instances. In the Wason selection task, checking the seven card (potential falsifier) is more informative than checking the three card (potential confirmer), though people systematically reverse this ordering due to confirmation bias.
  • Pluralist Bayesian Framework: No single probability function works for all arguments across contexts. Each scientific domain requires constructing appropriate probability models with context-specific assumptions and idealizations. Particle physics generates such powerful likelihood ratios that prior probabilities barely matter, while other sciences remain highly sensitive to priors, requiring explicit model construction for each case.

What It Covers

Philosopher Branden Fitelson explains how probability theory applies to scientific reasoning, distinguishing between objective physical probabilities and epistemic probabilities used to evaluate evidence strength, confirmation, and argument quality across different scientific contexts.

Key Questions Answered

  • Two-Dimensional Argument Strength: Strong arguments require both high probability (conclusion likely given premises) and high relevance (evidence actually changes probability). Diagnostic tests demonstrate this: a pregnancy test for a biologically male person shows high reliability but zero relevance, illustrating why both dimensions matter for rational belief updating.
  • Bayes Factors Over Posteriors: Scientific papers report likelihood ratios (true positive rate divided by false positive rate) rather than posterior probabilities because researchers cannot know readers' prior beliefs. This base factor represents objective, invariant information discoverable in laboratories, while posterior probabilities depend on individual background knowledge and assumptions.
  • Confirmation Versus Probability: Evidence can strongly confirm a hypothesis while leaving it improbable, or weakly confirm while making it highly probable. The base rate fallacy occurs when people confuse these dimensions: rare disease with reliable test yields low probability despite high confirmation, causing systematic reasoning errors in medical and scientific contexts.
  • Falsification Power: Popper's insight about falsification has quantitative validity: seeking counterexamples provides more confirmational power than seeking positive instances. In the Wason selection task, checking the seven card (potential falsifier) is more informative than checking the three card (potential confirmer), though people systematically reverse this ordering due to confirmation bias.
  • Pluralist Bayesian Framework: No single probability function works for all arguments across contexts. Each scientific domain requires constructing appropriate probability models with context-specific assumptions and idealizations. Particle physics generates such powerful likelihood ratios that prior probabilities barely matter, while other sciences remain highly sensitive to priors, requiring explicit model construction for each case.

Notable Moment

Fitelson reveals that Kahneman and Tversky's own research papers commit the same reasoning pattern they criticize in subjects: reporting base factors and likelihood ratios rather than posterior probabilities, implicitly acknowledging that scientists cannot determine how probable hypotheses are without knowing readers' prior beliefs and background knowledge.

Know someone who'd find this useful?

Episode Transcript

Hello, everyone, and welcome to the Mindscape podcast. I'm your host, Sean Carroll. One of the things that I always like to say about science and how it gets done is that science never proves things. This is something that is an important feature of science, especially in the modern world where what science does, how it reaches conclusions, how trustworthy it is, these are all under contestation by different parts of society. So it's important to understand what science is and how it actually reaches its conclusions. And the the claim that science never proves things, which is something that most scientists would go along with me on, comes from a comparison to real proof in mathematics or for that matter in logic. You know, most scientists have taken some math classes at least enough to know what it means to prove something in the old fashioned sense of Euclid and geometry or Aristotle and logic proving a conclusion from some well articulated premises. In the philosophical study of logic, this is known as deductive reasoning. You have some premises and you reach a conclusion. And science just doesn't go that way. Right? Science looks at the world. It looks at all sorts of things in the world, and it tries to figure out what the patterns are that the world follows. Always knowing that tomorrow, you might do a new experiment that will overturn your best guess as to what the pattern was, or maybe someone will do something as simple as just thinking of a better pattern. Right? A theoretical physicist coming up with a better idea for what the laws of physics really are. So if science doesn't prove things, if it just sort of comes closer and closer in some sense to getting it right, then what is what's going on? You know, one very common idea about what's going on is inductive logic rather than deductive logic. In inductive logic, we begin to see a pattern. You know, a, b, c, d, e, f, g. The next one is probably gonna be h. Right? Because we think that probably you're just mentioning the alphabet in alphabetical order. But there's all sorts of paradoxes that come up when you do inductive logic, like, how do you know that it's not a, b, c, d, e, f, z? That's a that's a sequence of letters that you that you could have. My old math teacher in college used to hate those SAT questions or standardized test questions that would give you a series of numbers and ask you to guess the next one. Because he said, I can I can make any number I want? I can come with a formula that would give you any number I want after the ones that you already showed me. So philosophers, unsurprisingly, are very interested in making as rigorous and careful as possible this idea of either induction or whatever should replace induction as the logic of understanding …

Get the full transcript (16,555 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Sean Carroll's Mindscape transcripts →

You just read a 3-minute summary of a 85-minute episode.

Get Sean Carroll's Mindscape summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

More from Sean Carroll's Mindscape

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Science Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's Software Engineering Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into Sean Carroll's Mindscape.

Every Monday, we deliver AI summaries of the latest episodes from Sean Carroll's Mindscape and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime