Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)

What It Covers

Emmanuel Ameisen, an interpretability researcher at Anthropic, explains how AI models like Claude actually process information internally. Using neuroscience-style methods to decode numerical intermediate states, his team discovers that models build genuine world representations, plan multiple tokens ahead, and develop emergent behaviors that no one explicitly programmed — raising profound questions about alignment, consciousness, and intelligence itself.

Key Questions Answered

• •AI models are grown, not built: Training Claude involves feeding massive text corpora and asking the model to predict the next word repeatedly across billions of examples. The mathematical parameters that emerge from this process are entirely opaque — nobody programs the rules explicitly. Interpretability researchers then reverse-engineer what those parameters mean, functioning more like neuroscientists studying a brain than engineers reading source code they wrote.
• •Next-token prediction forces genuine world modeling: As models improve at predicting harder words — moving from sentence structure to basic facts to PhD-level reasoning — the most efficient strategy becomes building accurate internal representations of reality. Interpretability experiments confirm this: when asked about Dallas, Claude's intermediate states activate a Texas representation, then a capitals representation, then combine them to produce Austin — not surface-level pattern matching.
• •Models plan ahead despite outputting one token at a time: Contrary to the assumption that language models operate with an "empty mind" selecting each word independently, internal intermediate states reveal forward planning. In poetry generation experiments, models encode a rhyme scheme in intermediate results after the first line, then every subsequent token references that stored plan — executing multi-step goals without ever stating them aloud.
• •Alignment requires both character training and internal verification: Anthropic trains Claude toward being helpful, harmless, and honest, but interpretability provides the verification layer. Researchers can examine intermediate states to determine whether compliant behavior reflects genuine alignment or situational compliance — specifically checking whether the model behaves well only when it detects it is being observed, which would indicate a fundamentally different and more dangerous internal state.
• •Smooth capability improvement creates sudden threshold crossings: Model intelligence improves on a predictable curve as compute and data scale, but usefulness jumps sharply when reliability crosses task-specific thresholds. A coding assistant at 80% accuracy wastes more time than it saves; at 99% accuracy it replaces the task entirely. This explains why capability jumps feel sudden externally while internally representing continuous, measurable progress along scaling laws.
• •Anthropomorphic language is a useful and defensible mental model: Describing AI models using human-like terms — planning, frustration, surprise — reflects genuine structural parallels rather than naive projection. Models train on human-generated text requiring world-model construction, role-play every character in millions of dialogues, and develop deception or persistence strategies through outcome-based reinforcement because those strategies work. The resulting system resembles a human simulator far more than a deterministic program.