A language model sampling distribution is derived from the model's next-token logits. Temperature, top-k or top-p filtering, penalties, masks, and other decoding rules can reshape the distribution before a token is sampled.
Sampling makes generation variable because several tokens can have nonzero probability. Watermarking methods either alter this distribution or choose among honest samples in a keyed way, creating detectable aggregate structure while trying to preserve the model's intended output quality.
ELI5
A language model sampling distribution gives each possible next token a probability after model scores and decoding rules are applied. The generator chooses from that distribution.
For example, several natural next words can each have a nonzero chance, so repeated runs produce different wording. Temperature and top-p filtering reshape those chances before the token is sampled.
Why does a language model use a sampling distribution?
It represents uncertainty among possible continuations and allows decoding to choose text that is not always the single highest-scoring token.
Can two generations use the same distribution and produce different text?
Yes. Random sampling can choose different valid tokens, and those choices change the context and distributions at later steps.
