A language model produces one logit for each token in its vocabulary at a generation step. A transformation such as softmax converts the logits into a probability distribution, after optional controls such as temperature, masking, repetition penalties, or watermark bias are applied.
Changing a logit changes relative token probability without rewriting the token itself. Soft watermarking exploits this property by adding a small bias to keyed preferred tokens, increasing their aggregate frequency while leaving nonpreferred choices available.
Acronyms and aliases
LLM logit acronymtoken logit variant
Related terms
Frequently asked questions
Is a language model logit a probability?
No. It is an unnormalized score that becomes part of a probability distribution after a normalization step such as softmax.
How does logit bias affect generated text?
It raises or lowers selected token scores, changing how likely those tokens are to be sampled at each generation step.
