Building Turbopuffer: Simon Hørup Eskildsen on Vector Search for AI

AI Engineer56m 30s
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Video summary

    In conversation with Gergely Orosz, Turbopuffer founder Simon Hørup Eskildsen traces a path from large-scale infrastructure work at Shopify to building a lower-cost search database. A document-search project after ChatGPT's launch and a recommendation prototype for Readwise exposed vector-storage costs that, by his account, exceeded what the customer could justify. That experience pushed him toward storing vector data in S3 and keeping frequently queried data in faster cache layers.

    Simon Hørup Eskildsen describes the engineering tradeoff behind that design: object storage is durable and inexpensive, but its tail latency makes naive search too slow. The initial Turbopuffer implementation used clustered files and a simple cache, then evolved as real workloads arrived. The account is useful for understanding why storage layout, p99 latency and cache behavior matter to retrieval systems that support AI products.

    Cursor became Turbopuffer's first customer after a public launch. Simon Hørup Eskildsen says the migration cut Cursor's relevant database bill by 95 percent, while Gergely Orosz recounts the trust-building work that preceded the deal. That percentage is a firsthand claim in the interview, not an independently audited comparison. The conversation also illustrates how a small infrastructure vendor can be tested against the reliability and cost needs of a fast-growing AI coding product.

    The later discussion turns to compute supply. Simon Hørup Eskildsen recounts a meeting with Nvidia leadership, explains why Turbopuffer primarily uses CPUs and SSDs, and argues that reinforcement-learning environments and deployed agents are competing for CPU capacity alongside GPU-heavy model workloads. These are his observations and forecasts, not a measured industry-wide capacity study.

    The interview ends with Turbopuffer's approach to fundraising and distributed-team culture. Those sections are general company-building material rather than AI analysis, so the catalogue value rests chiefly on the concrete vector-search and AI-infrastructure passages.

    Original YouTube thumbnailWatch on YouTube