What is prompt sensitivity?

Definition

Artificial intelligence prompt sensitivity occurs when wording, order, examples or surrounding context substantially affect model behavior. Casual prompts may understate capability, while highly tuned prompts can overstate how well the model handles ordinary use.

Evaluation should use several realistic prompt forms and preserve the harness and settings. Consistent performance across variations provides stronger evidence than one carefully selected prompt that produces an impressive answer.

Acronyms and aliases

prompt-dependent model behavior synonymAI prompt sensitivity variantartificial intelligence prompt sensitivity variant

Frequently asked questions

Why are AI models sensitive to prompts?

Models condition their output on token patterns, so wording, examples, order and context can change the inferred task.

How should prompt sensitivity be evaluated?

Repeat representative tasks with controlled prompt variations and compare success, quality and variance.

Videos explaining prompt sensitivity