Distill the LLM, Don't Serve It: Search & Personalization at DoorDash - Raghav Saboo, DoorDash

AI Engineer22m 10s
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Video summary

    Raghav Saboo argues that marketplace discovery depends on understanding both what an item means and what a shopper intends. A puppy-shopping journey illustrates why a single query can span search, recommendations and several sessions, motivating four shared approaches: supervision, catalogue semantics, shopper memory and steerable content generation.

    For relevance supervision, the team combines human-labeled query and item pairs with stronger language-model review of suspicious cases. It then uses an offline labeler to produce graded relevance targets for retrieval and ranking. A two-stage contrastive retriever mines hard negatives, while the ranker adds a relevance objective alongside engagement and conversion objectives, keeping the online serving path in the existing stack.

    DoorDash also assigns hierarchical semantic IDs to catalogue items, giving models finer item relationships than a fixed taxonomy. Shared prefixes can connect related products, support new and sparsely observed items, and help audit catalogue labels. The speaker reports improvements in retrieval and ranking, but the numerical results are not independently verified here.

    For personalization, the system represents shopper preferences and current context as human-readable memory blocks, embeddings and graph relationships that different models can reuse. An offline language-model process combines these representations with semantic IDs to prepare personalized store collections, while existing retrieval and ranking systems select and serve the items.

    Original YouTube thumbnailWatch on YouTube