AI Trends 2026: OpenClaw Agents, Reasoning LLMs, and More with Sebastian Raschka - #762

What It Covers

Sebastian Raschka, independent LLM researcher, joins Sam Charrington to assess the LLM landscape in early 2026. They cover reasoning model advances, inference-time scaling techniques, the rise of agentic tools like OpenClaw, practical workflow automation using LLMs, and what to expect from post-training research through the rest of 2026.

Key Questions Answered

• •Post-training vs. pre-training R&D shift: Research teams are now concentrating resources on post-training techniques rather than pre-training because low-hanging fruit remains in reinforcement learning and reasoning pipelines. Pre-training is already highly optimized — more data and better data mixes yield diminishing returns — while post-training algorithms like GRPO still have significant room for improvement through relatively accessible algorithmic tweaks.
• •Verifiable rewards as the reasoning engine: DeepSeek R1's breakthrough relied on training models using math and code problems where correctness can be verified deterministically — using tools like SymPy for symbolic math comparison or code compilers. This eliminates the need for human evaluators, enabling generation and scoring of tens of thousands of answers cheaply. Extending verifiable rewards to domains like drug design or protein structure modeling is the next frontier.
• •Inference-time scaling via self-consistency and self-refinement: Two concrete techniques boost model accuracy without retraining. Self-consistency generates multiple answers at varied temperatures and selects via majority vote (best-of-N). Self-refinement feeds a model's output back to itself or another model with a rubric, prompting iterative correction. DeepSeek Math 3.2 demonstrated that cranking up both techniques enabled gold-level competition math performance from the same base model.
• •LLMs as tool-builders, not just task-doers: The highest-leverage use of LLMs for technical users is building deterministic workflow tools — native apps, scripts, custom web tools — rather than using LLMs for every task directly. Raschka built macOS apps for podcast chapter-mark insertion and metadata extraction from arXiv links. Charrington built a podcast analytics pipeline. Using LLMs to create deterministic tools avoids hallucination risk on repetitive structured tasks.
• •Agentic systems require model fine-tuning for multi-agent environments: Current agentic tools like OpenClaw and Claude Code use standard LLMs not specifically trained for multi-agent interaction. OpenAI's Codex backend is a fork of GPT-5.3 fine-tuned specifically for agentic coding tasks. Raschka predicts major labs will fine-tune dedicated agent models for multi-agent settings, similar to how Codex diverged from the base model, improving reliability in looped, tool-using pipelines.
• •Mixture-of-experts and multi-head latent attention define 2025-2026 architecture trends: DeepSeek V3's architecture — combining mixture-of-experts with multi-head latent attention (MLA) — became the dominant template, adopted by Kimi (scaled to 1 trillion parameters) and Mistral AI. MLA compresses key-value cache via low-rank projection (similar to LoRA), trading compute for memory efficiency. DeepSeek's sparse attention mechanism further reduces quadratic scaling costs, making these the practical production-proven architectural choices to watch.