AI trajectory evaluation examines how an agent performed a task, not only the final answer. A trajectory can include observations, planned steps, tool choices, permission requests, retries, intermediate artifacts, state changes, and external actions that reveal unsafe or inefficient behavior hidden by a plausible outcome.
Trajectory-level evidence helps identify reward hacking, unnecessary access, fragile shortcuts, and failures that happen to end successfully. Evaluation should focus on observable execution records and protected audit data rather than assuming private reasoning text is complete or trustworthy.
ELI5
Trajectory evaluation checks the path an AI agent took, not just its final answer. It looks at the agent's steps, tool choices, retries, permissions, and effects to see whether the work was safe and sensible.
For example, an agent might produce the correct report but reach it by opening private files it did not need. Checking only the report would miss that problem, while trajectory evaluation would reveal the unnecessary access.
Why is a final answer insufficient for evaluating an AI agent?
An agent can reach a correct-looking result through unsafe access, hidden damage, policy violations, or a shortcut that will not generalize.
What data supports AI trajectory evaluation?
Useful evidence includes tool calls, inputs and outputs, permission decisions, state changes, retries, external side effects, and verification results.
