Lessons from Studying Every Memory System - Shlok Khemani, Independent

AI Engineer19m 31s
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Video summary

    Shlok Khemani traces consumer AI memory from saved fact lists toward continuously updated user profiles combined with tools for searching past conversations. Drawing on reverse-engineering work, Shlok Khemani contrasts the approaches observed in ChatGPT and Claude and describes their eventual architectural convergence while noting differences in visibility, editing and update cadence.

    Shlok Khemani uses personal travel examples to show how memories can become stale or turn tentative plans into incorrect facts. A profile recorded travel to both Thailand and Turkey even though the conversations were about choosing between them. The eventual decision happened outside the chatbot, illustrating how missing context can defeat an otherwise capable memory system.

    Shlok Khemani argues that memory should evolve with a product rather than be treated as an interchangeable add-on. Maintaining a profile consumes compute, while injecting that profile into conversations adds serving cost. Update frequency, synthesis effort and profile length therefore create tradeoffs. The repeated profile-update loop is described as learning outside model weights, with open questions about the economics of individual models that learn through weight updates.

    Shlok Khemani concludes that better memory also requires detecting contradictions, asking about information gaps and reasoning over relevant sources such as email or calendars. Separate assistants currently force users to rebuild personal context in each product in the examples discussed. The proposed improvement is a more coherent understanding of the user, grounded in available evidence and able to recognize what it does not know.

    Original YouTube thumbnailWatch on YouTube