Brendan Rappazzo presents Morgan Stanley's AlphaLab, a provider-agnostic system for research tasks such as forecasting and model optimization. It combines contextual research, evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. construction and repeated experiments rather than relying on a single agent to complete the whole process.
Research agents collect notes, evaluation builders work with conceptual and programmatic critics, and a strategist coordinates worker experimentsAgent orchestration coordinates AI agents, tools, people, tasks, state, and control flow so a larger workflow reaches a verified outcome. through a Kanban-style queue and Slurm compute jobs. Human researchers can steer the queue and compare candidate models on held-out dataAn evaluation set is a collection of examples kept for measuring an AI system rather than training it.. Reported benchmark and internal-model improvements are presented as the speaker's results, not independent validation or investment advice.
Rappazzo describes harnessAn AI agent harness is the software framework that packages a model with tools, instructions, context management, execution controls, and user interaction. design and measurement errors as recurring problems. He argues that carefully constructed evaluation environments encode scarce enterprise expertise, support measurable iteration and can provide feedback for improving the harness itselfAn AI feedback loop occurs when AI outputs or their consequences become inputs that influence later model or agent behavior.. The presentation closes with plans for automated harness optimization and learning from successful research traces.
Watch on YouTube




