A dense artificial intelligence model uses the same connected parameter set for every token processed through a layer. This makes its computational path straightforward, but a large dense model may require substantial memory bandwidth and arithmetic work for every generated token.
Dense models contrast with mixture-of-experts models, which route a token through selected expert blocks. Parameter count alone therefore does not provide a complete comparison of memory needs, active compute or inference speed across the two designs.
Acronyms and aliases
dense AI model acronymdense model variant
Related terms
Frequently asked questions
How is a dense model different from a mixture-of-experts model?
A dense model activates its full parameter path for each token, while a mixture-of-experts model selects only some expert components for a given token.
Does a dense model use every stored parameter for every token?
In the usual architecture-level sense, its layers use the complete dense path, unlike sparse expert routing that activates only selected parameter blocks.
Videos explaining dense artificial intelligence model
