Many model requests share a stable prefix such as system instructions, tool definitions or repository guidance. A provider or application can cache the processed representation of that prefix and reuse it when later requests contain the same content.
Caching is most effective when the stable portion is large and repeated frequently. Systems still need invalidation and privacy rules so changed instructions are not served from stale state and one user's sensitive context is not reused for another.
Acronyms and aliases
cached prompts variantprompt cache variant
Related terms
Frequently asked questions
How does prompt caching reduce artificial intelligence cost?
It avoids processing the same stable prompt prefix from the beginning on every compatible model request.
What content is suitable for prompt caching?
Stable repeated content such as system instructions, tool definitions or shared project context is usually the best candidate.
