Maor Bril argues that frame-level metricsAn AI evaluation metric is a defined measurement used to quantify a particular aspect of an AI system's performance. do not establish whether a video tells the intended story. His Character.ai workflow evaluates consistency, physics, pacing and audio alignment, starting with repeatable metrics and human-calibrated model judges.
Maor Bril distillsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model. those judgments into a smaller vision-language modelA vision-language model jointly processes images and language so it can describe, answer questions about or act on visual information. to make evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. fast enough for the generation loop. Pairwise comparisons replace arbitrary absolute scores, but an initial dataset teaches shortcuts: the model rewards visual coherence rather than the intended quality dimensions.
Maor Bril describes repairing the training dataTraining data is the collection of examples and signals used to adjust an AI model's parameters so it learns useful patterns. with consistent encoding and annotation, avoiding a simple real-versus-AI detector. Agents then use the evaluator to detect and repair drift earlier. In the Q&A, he discusses human taste, serving economics and audio timing, while acknowledging that lip-sync evaluation remains unresolved.
Watch on YouTube




