Distill the LLM, Don't Serve It: Search & Personalization at DoorDash - Raghav Saboo, DoorDash

AI Engineer22m 10s
Raghav Saboo in a blue shirt beside the headline “DISTILL THE LLM” on a black background.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Raghav Saboo argues that marketplace discovery depends on understanding both what an item means and what a shopper intends. A puppy-shopping journey illustrates why a single query can span search, recommendations and several sessionsA recommendation system ranks or selects items for a user by estimating which options are most relevant to their interests, intent or context., motivating four shared approaches: supervision, catalogue semantics, shopper memory and steerable content generation.

    For relevance supervision, the team combines human-labeled query and item pairs with stronger language-model review of suspicious cases. It then uses an offline labeler to produce graded relevance targets for retrieval and rankingKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model.. A two-stage contrastive retrieverContrastive learning trains an AI model to place matching examples closer together and nonmatching examples farther apart in its learned representation. mines hard negatives, while the ranker adds a relevance objective alongside engagement and conversion objectives, keeping the online serving path in the existing stack.

    DoorDash also assigns hierarchical semantic IDs to catalogue itemsA semantic ID is a compact identifier that gives an AI system clues about an item's meaning or similarity to other items., giving models finer item relationships than a fixed taxonomy. Shared prefixes can connect related products, support new and sparsely observed items, and help audit catalogue labels. The speaker reports improvements in retrieval and ranking, but the numerical results are not independently verified here.

    For personalization, the system represents shopper preferences and current context as human-readable memory blocksAI personalization adapts information, recommendations, interfaces, or agent behavior to an individual or group using permitted context and observed needs., embeddings and graph relationships that different models can reuse. An offline language-model process combines these representations with semantic IDs to prepare personalized store collections, while existing retrieval and ranking systems select and serve the items.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows0
    • Coding & tools0
    • Models & learning100
    • Infrastructure & compute75
    • Research & evaluation75
    • Safety & security0
    • Business & productivity100
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    Raghav Saboo in a blue shirt beside the headline “DISTILL THE LLM” on a black background. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 25 September 2026 and duration 22m 10s.

    Raghav Saboo explains how DoorDash uses LLMs offline to label relevance, learn catalogue semantics and shopper memory, then serves faster search and personalized collections through existing models.