What is context pruning in AI?

Definition

Long workflows can accumulate messages, retrieved documents and tool results that no longer help the current decision. Context pruning selects the information that remains relevant and omits repetition, stale state or detail that can be summarized safely.

Good pruning reduces token use and distraction without hiding critical constraints or evidence. It needs task-aware relevance, provenance and conservative handling of permissions, decisions and facts that later steps may still require.

ELI5

Context pruning removes information that an AI no longer needs for its current task. It keeps the useful instructions and evidence while dropping repetition, stale details and tool output that would only take up space.

For example, a long research session may contain ten copies of the same search result and several abandoned plans. Pruning can keep one reliable result and the current plan, but it should preserve permissions, decisions and source details that later steps may still need.

Why prune an AI model's context?

Pruning can reduce token cost, latency and distraction from irrelevant or duplicated information.

What is the risk of context pruning?

Removing a critical requirement, decision or piece of evidence can make later model output incorrect or unsafe.

  1. Mingsheng Hong beside the headline Measure Value Not Tokens in attention blue and white
Definition card for context pruning: What is context pruning in AI?

Context pruning removes low-value or redundant information from the input supplied to an AI model while preserving evidence needed for the task.