Ashwin Paranjape explains why his team chose quarterly earnings forecasts rather than stock prices to test financial agents: reported results provide numerical outcomesEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. and analyst estimates provide a comparison.
The team built a point-in-time environment that restricts information to what was available before each forecastBacktesting applies a strategy to historical data to estimate how it would have behaved under past market conditions.. Ashwin Paranjape reports results across more than 450 earnings events and describes comparisons with both consensus and a bias-corrected baselineA benchmark is a defined test or collection of tasks used to compare a system's performance under specified conditions..
Examples show an agent finding supplier evidence in one forecast and failing to act on relevant evidence in another. Ashwin Paranjape also discusses tests of data recency, expert guidance and withholding consensus. These are the presenters reported experiments, not evidence of investment performance.
Watch on YouTube
