Károly Zsolnai-Fehér presents Qwen 3.8 Flash Next as a mixture-of-experts model designed to use only a small portion of its parameters for each token. That structure makes it attractive for machines with ample memory but comparatively limited memory bandwidth, while early local tests show practical generation speeds.
Károly Zsolnai-Fehér highlights Qwen Sparse Attention, which groups tokens into small blocks before searching for the most relevant context. This avoids the quadratic growth of full attention and can make increasingly long conversations or document collections cheaper to process.
Károly Zsolnai-Fehér also explains gated residual branches that protect information from being repeatedly overwritten across layers, plus an engram embedding layer that retrieves common short token combinations cheaply. Together, these mechanisms help the model compete with much larger open-weight systems while remaining more efficient to run.
Watch on YouTube
