Olive Song describes MiniMax M3 as a roughly 400-billion-parameter mixture-of-experts model with about 20 billion active parameters, a functional one-million-token context windowA context window is the maximum amount of tokenized information an AI model can consider during one processing session. and native coding, agentic and multimodal capabilities. Thomas Wolf connects those design choices to the practical demands placed on modern AI agents.
Song argues that short context windows break down when agents must combine long conversations, tool responses, presentation decks, reports and video. MiniMax Sparse AttentionSparse attention reduces transformer computation by allowing each token to attend to a selected subset of other tokens instead of comparing with every token. reduces the cost of processing that context, while multimodal training lets the model reason over different source formats instead of relying on a text-only conversion layer.
The discussion also covers internal research agents that automate model experiments, data work and kernel optimization. Song expects future systems to coordinate multiple specialized agentsA multi-agent system contains multiple AI agents that interact, coordinate, divide work, or influence one another while pursuing tasks. or models, with routing and shared context becoming as important as the capability of any single model.
Watch on YouTube
