Tim Sweeney shows an agent launching training experiments, comparing earlier runs and producing research summaries and visualizations. The system combines a chat interface with code execution and long-running GPU jobs. The live batch does not beat the earlier best loss, illustrating iteration rather than guaranteed improvement.
Tim Sweeney outlines a worker harness linked to a sandbox, model provider, job queue and observability layer. Production conversations feed an offline improvement loop: the team examines traces, identifies behavioral failures and turns them into reusable tasks that test proposed changes before release.
Tim Sweeney describes evaluations combining model judges for correctness and useful insights with rule-based limits on tool calls. Human reviewers remain necessary because automated judges miss behavioral nuances. Tim Sweeney recommends domain context and useful tools before elaborate memory or harness engineering, and treating evaluation results as explicit release decisions.
Watch on YouTube
