Ep. 11 - The Brain Behind the Bot: Decoding AI's Hidden World

Welcome to "In the Long Run," the podcast where we explore technology, data, and AI decision-making.

In this episode, Jim and João discuss Anthropic's research on mechanistic interpretability, which aims to reverse-engineer neural networks to understand how AI models actually work. They compare AI models to human brains as "black boxes" with unclear internal processes and explore how multilingual models like Claude may use a universal "concept language" internally before translating to specific human languages. The conversation also touches in the part of the study that reveals evidence that AI models plan multiple words ahead rather than simply predicting one word at a time.

We further discuss the shift from "large language models" to "foundation models" as these systems now incorporate multiple modalities including text, images, and audio. They highlight GPT-4's new image generation capabilities that recently dominated social media with "Ghiblifying" images. The episode concludes with AI news including Google's Gemini 2.5 release, OpenAI's plans for an open-weight model, and the adoption of Model Context Protocol as a potential industry standard for AI tools.