Tim Scarfe interviews Ilia Shumailov and Alexander Panfilov about a weakness in the encrypted reasoning state returned by several frontier-model APIs. The state is meant to let a conversation resume or fork without revealing the hidden chain of thought, but the researchers found that it can be replayed outside its original context.
A smaller related model can process a captured reasoning blob and repeat the hidden content in plain text. That can expose sensitive details preserved in a shared conversationData privacy governs how personal, confidential, or sensitive information is collected, used, shared, retained, and protected in AI systems. even when the visible prompt and answer have been sanitized, and it complicates chain-of-thought monitoringChain-of-thought monitoring analyzes a reasoning model's exposed intermediate reasoning for signs of errors, policy violations, deception, or unsafe plans. because some traces use terse or unusual internal language.
The same portability can place fabricated reasoning into another conversation or a downloaded agent trace. A resumed model may then treat the injected state as its own prior thought, creating a path for jailbreaks, prompt injectionPrompt injection occurs when untrusted content attempts to redirect an AI system's behaviour beyond the authority of that content. and poisoned long-running agent workflows.
The discussion separates the demonstrated replay and jailbreaking risk from broader claims about model distillation, which the evidence does not prove. Proposed defenses include keeping reasoning state server-side, binding it to a user, model and conversationNetwork security protects communications infrastructure and connected systems from unauthorized access, disruption and misuse., blocking repeated state, detecting leaked reasoning patterns and testing mitigations in controlled experiments.
Watch on YouTube
