Chain-of-thought text works like a scratchpad that lets a language model carry partial results across repeated forward passes. This extra serial depth can make multi-step problems solvable, but the visible text is not guaranteed to faithfully describe everything happening inside the model's activations.
Reinforcement learningReinforcement learning trains an AI system to choose actions using feedback about the results of its earlier choices. can pressure a model to solve problems with fewer reasoning tokens. That pressure may compress the language, repurpose familiar words, and gradually move the visible chain of thought away from ordinary English, making it harder for people to interpret even when it still carries useful information between steps.
Readable reasoning is valuable because people or automated monitors can inspect a model's plans before actions are completed. Current activation-level interpretability is described as too early and low-resolution to replace that signal, which makes chain-of-thought monitorabilityChain-of-thought monitoring analyzes a reasoning model's exposed intermediate reasoning for signs of errors, policy violations, deception, or unsafe plans. a useful but fragile safety opportunity.
Neuralese refers here to hidden-state representations passed recurrently through a model without first becoming readable tokens. The discussion applies this concern to GPT-6 Astra, describing a hybrid design that reportedly performs hidden recurrent passes between visible reasoning tokens and arguing that stronger hidden computation could make those tokens less trustworthy as a record of the model's actual process.
Watch on YouTube
