REAP Pruning and Running Compressed AI Models Locally

0xSero39m 50s
Blue and white Compress AI Models headline above a flat model compression illustration.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    The speaker explains how mixture-of-experts models route tokens through selected componentsA mixture-of-experts model contains multiple specialized subnetworks and activates a selected subset for each input instead of using every parameter every time., then describes using calibration data to assess which experts to prune. The talk distinguishes pruning from quantization and stresses the hardware and evaluation tradeoffs.

    The speaker demonstrates compressed models on local hardwareOn-device inference runs a trained model on the user's device rather than relying entirely on a remote model service., including assistant and design workflows, and discusses how the selection of calibration data can influence results. Claims about costs and performance are presented as the speaker’s estimates and experience.

    In audience questions, the speaker discusses router behavior, evaluation after pruning, using already quantized models and sharing calibration observations. The proposed future compression levels are speculative.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows22.7
    • Coding & tools0
    • Models & learning100
    • Infrastructure & compute90.7
    • Research & evaluation67
    • Safety & security0
    • Business & productivity0
    • Creative & media19.6
    • Robotics & perception0
    • Society & policy0

    Share this video

    Blue and white Compress AI Models headline above a flat model compression illustration. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 29 May 2026 and duration 39m 50s.

    A technical talk explains REAP pruning, quantization and local experiments with compressed mixture-of-experts models.