A system can perform well on familiar examples by exploiting incidental cues or adapting too closely to repeated feedback. This can affect model training as well as prompts, retrieval choices and an agent's evaluation workflow.
Held-out examples help assess whether the method works beyond the cases used to develop it. Reliable conclusions also need representative data and appropriate separation between tuning and final evaluation.
ELI5
Overfitting is doing well on the examples you practiced but not on genuinely new ones. The system may have learned a shortcut rather than the useful general rule.
For example, an assistant might rely on a company-name pattern that works in a small test list. Trying unseen companies can reveal whether that shortcut actually helps.
Can an agent workflow overfit without changing model weights?
Yes. Prompts, rules and evaluation procedures can be tuned too closely to a known set of examples.
Why use held-out cases?
They provide evidence about behavior on examples that did not guide the method's development.




