A model-based grader receives an output plus a rubric, reference or comparison set and returns a judgment. It can scale evaluation beyond manual review and can assess qualities that are difficult to express with a simple exact-match metric.
The grader is itself a model and can be biased, inconsistent or sensitive to wording and presentation. Reliable use includes calibration against human judgments, repeated checks, clear rubrics and safeguards against an evaluated system exploiting the grader's preferences.
ELI5
A model-based grader is an AI used to judge the work produced by another system. It follows a scoring guide, much like a reviewer using a checklist.
For example, one model could score whether a generated image follows a written request and keeps the subject consistent. The score can speed up testing, but people should compare the grader with human opinions because the grader can misunderstand the work too.
Why use a model-based grader?
It can evaluate large numbers of outputs quickly and apply a detailed rubric where simple automatic metrics are insufficient.
Can a model-based grader be wrong?
Yes. It can have biases, miss context, respond inconsistently or reward superficial patterns, so it should be calibrated and audited.
