What is soft language model watermarking?

Definition

Soft language model watermarking adds a controlled bias to the logits of green-listed tokens before sampling. A strongly preferred factual continuation can still win even when it is not green, while uncertain choices are gently shifted toward tokens that contribute to the hidden pattern.

This approach preserves quality better than a hard ban but creates a weaker signal. Reliable detection usually needs enough tokens and a statistical test that compares the observed green-token count with the amount expected under unwatermarked sampling.

ELI5

Soft language model watermarking gently makes certain secret-keyed token choices more likely during generation. It does not ban other words, so an important or clearly better continuation can still be selected.

For example, when several next tokens fit almost equally well, the model slightly favors one from a hidden preferred set. A detector needs enough text to decide statistically whether the preferred pattern appears more often than chance.

How does soft language model watermarking preserve answer quality?

It biases preferred tokens rather than banning alternatives, so a high-probability correct continuation can still be selected.

Why does soft watermarking need statistical detection?

Individual token choices remain plausible, so the signal appears as a small aggregate preference across many generation steps.

  1. Flat token cards and a magnifying glass beside the words Why AI Watermarks Stay Hidden
Definition card for soft language model watermarking: What is soft language model watermarking?

Soft language model watermarking increases the probability of keyed preferred tokens without completely forbidding other valid choices.