Károly Zsolnai-Fehér presents Kimi K3 as a large open-weights modelAn open-weight model makes its trained parameter values available so others can run or adapt it under the release's licence. that approaches frontier coding capability while remaining downloadable and available through a lower-cost API. The model has 2.8 trillion parameters, so most people cannot run the full version locally, but its pricing and future distilled variantsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model. could still affect the wider market.
Károly Zsolnai-Fehér highlights two technical changes. Kimi Delta Attention maintains a compact, gradually updated memory instead of repeatedly processing all prior context, while attention residuals preserve useful intermediate representationsA latent representation is an internal numerical encoding an AI model uses to represent and process information. across model layers.
The developers report about 2.5 times more scaling efficiency than Kimi K2, meaning greater learning progress from the same training computation rather than a direct 2.5 times speed or cost improvement. Károly Zsolnai-Fehér concludes that open weights, published techniques and lower inference prices can spread improvements across other open models while increasing price pressure on closed providers.
Watch on YouTube
