One way to measure it is to group tasks by how long skilled people typically take to complete them, then estimate how the system's success rate changes with task length. The measure depends on the tasks, allowed tools and the definition of success.
A longer measured horizon can indicate greater reliability on extended work. Scores may change when evaluators tighten checks on whether a result was actually achieved.
ELI5
For example, if an assistant completes most of the tasks that take a person ten minutes but fails most hour-long tasks, an evaluation can estimate where its success starts to fall.
That estimate is about the tested work. It does not tell you how long the assistant can work independently on every subject.
Is a task horizon measured by how long the model runs?
In this evaluation approach, task length comes from a human completion-time reference. The model's own runtime is a separate measure.
Why specify a success rate?
Different thresholds produce different horizons. Specifying the threshold makes the result interpretable and easier to compare.

