An AI model embedding layer maps each token or input unit to a learned vector. Those vectors provide the continuous numerical representation consumed by later model layers.
A large vocabulary or embedding dimension can make the component consume substantial memory even when it does not dominate per-token arithmetic. Some local runtimes can place the embedding weights in system memory to preserve accelerator capacity for other layers.
ELI5
A model embedding layer turns token identifiers into lists of numbers that later AI model layers can process. The learned numbers place tokens with useful relationships into a mathematical representation.
For example, when a model reads a word token, the embedding layer looks up its vector before attention and other layers work with it. A large vocabulary can make this lookup table use substantial memory even when the lookup itself is simple.
What does an embedding layer do with a text token?
It looks up a learned numerical vector for the token, giving later model layers a representation they can transform.
Why can a model embedding layer be memory-heavy?
It stores a vector for every supported token or input unit, so a large vocabulary and wide vectors create many parameters.
