A language model sampling distribution is derived from the model's next-token logits. Temperature, top-k or top-p filtering, penalties, masks, and other decoding rules can reshape the distribution before a token is sampled.
Sampling makes generation variable because several tokens can have nonzero probability. Watermarking methods either alter this distribution or choose among honest samples in a keyed way, creating detectable aggregate structure while trying to preserve the model's intended output quality.
Acronyms and aliases
LLM sampling distribution acronymnext-token probability distribution variant
Related terms
Frequently asked questions
Why does a language model use a sampling distribution?
It represents uncertainty among possible continuations and allows decoding to choose text that is not always the single highest-scoring token.
Can two generations use the same distribution and produce different text?
Yes. Random sampling can choose different valid tokens, and those choices change the context and distributions at later steps.
Videos explaining language model sampling distribution
