Annabell Schäfer argues that automated prompt improvement depends on a well-defined objective and domain expertise. Her classification example begins with expert categories and labeled paper titles and abstracts, rather than expecting an optimizer to discover the right goal from vague instructions.
Annabell Schäfer separates fitting, validation and test data and uses a stronger model to suggest changes for a cheaper classifier. The reported example improves over its baseline, but the held-out score is lower than the best validation result. Ambiguous labels and the limited task matter when interpreting those numbers.
Annabell Schäfer recommends inspecting failures, improving examples and rules, and setting stopping criteria instead of optimizing indefinitely. Continued production review is needed when inputs change. The demonstration illustrates a workflow for measurable improvement, not a guarantee that an automatically revised prompt generalizes to every application.
Watch on YouTube
