Theo Browne describes a design experiment in which several AI models reviewed successive screenshotsA model-based grader uses an AI model to evaluate, compare or score outputs according to defined criteria. of an application and proposed changes. He explains how comparing the interface with other products changed the reviewers’ scores and how typography, hierarchy and contrast were refined.
Theo Browne cautions that model scores vary with context and may be influenced by the target score included in a promptPrompt engineering is the practice of designing and refining instructions, context, examples, and constraints to obtain useful AI outputs.. He discusses the time and estimated cost of repeated review rounds while treating the results as his experience rather than a controlled comparison.
He also describes building an evaluation from past tool-use failuresAn evaluation set is a collection of examples kept for measuring an AI system rather than training it. and iterating on tool descriptions to reduce context consumptionContext engineering designs how relevant instructions, evidence, state and tool information are assembled for a model's task. while maintaining evaluation performance. He argues that measurable feedback can guide agent-driven improvementAn AI feedback loop occurs when AI outputs or their consequences become inputs that influence later model or agent behavior..
Watch on YouTube
