A historical engineering benchmark reconstructs earlier project states from code, logs, messages, documents, tickets, and other evidence. The model receives a pre-solution snapshot and must diagnose or solve the issue without access to later fixes.
This design can approximate real information constraints and reduce future leakage. It still needs clean timestamps, representative issue selection, objective grading, preserved tool conditions, and checks that later solution details did not enter the model's context or training data.
ELI5
A historical engineering benchmark asks an AI to solve a real technical problem from the past. It gives the model only the information engineers had before they found the answer.
For example, the test can restore a repository, logs, and messages from the day before a bug was fixed. The AI must propose a solution without seeing the later patch, which makes the test closer to the original challenge.
