Evaling Video Slop: Maor Bril, Character.ai

AI Engineer23m 13s
Maor Bril beside the blue-and-white headline “EVALUATING AI VIDEO” on a black background.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Maor Bril argues that frame-level metricsAn AI evaluation metric is a defined measurement used to quantify a particular aspect of an AI system's performance. do not establish whether a video tells the intended story. His Character.ai workflow evaluates consistency, physics, pacing and audio alignment, starting with repeatable metrics and human-calibrated model judges.

    Maor Bril distillsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model. those judgments into a smaller vision-language modelA vision-language model jointly processes images and language so it can describe, answer questions about or act on visual information. to make evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. fast enough for the generation loop. Pairwise comparisons replace arbitrary absolute scores, but an initial dataset teaches shortcuts: the model rewards visual coherence rather than the intended quality dimensions.

    Maor Bril describes repairing the training dataTraining data is the collection of examples and signals used to adjust an AI model's parameters so it learns useful patterns. with consistent encoding and annotation, avoiding a simple real-versus-AI detector. Agents then use the evaluator to detect and repair drift earlier. In the Q&A, he discusses human taste, serving economics and audio timing, while acknowledging that lip-sync evaluation remains unresolved.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows25
    • Coding & tools0
    • Models & learning75
    • Infrastructure & compute25
    • Research & evaluation100
    • Safety & security0
    • Business & productivity10
    • Creative & media100
    • Robotics & perception25
    • Society & policy0

    Share this video

    Maor Bril beside the blue-and-white headline “EVALUATING AI VIDEO” on a black background. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 25 July 2026 and duration 23m 13s.

    Maor Bril explains how to evaluate generated videos for story, physics, pacing and audio, then place a fast judge inside the generation loop.