Homa: The End of TCP for AI Clusters - John Ousterhout, Stanford

AI Engineer18m 48s
John Ousterhout in a dark shirt on a black background beside the blue-and-white headline HOMA FOR AI CLUSTERS.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    John Ousterhout argues that inferenceInference is the use of a trained model to produce an output from supplied input. and agentic workloads increasingly depend on short coordination messages, even while training still moves large amounts of data. When distributed GPU jobs must synchronize between compute phases, a slow message can leave processors idle and limit overall throughput.

    The talk traces tail latencyTail latency measures the slowest fraction of requests, commonly expressed with high percentiles such as p95, p99 or p99.9. to incast, where large transfers fill a switch queue and short messages wait behind them. It also explains why sender-side congestion control reacts after network feedback and why byte streams make it difficult to prioritize individual short messages.

    John Ousterhout presents Homa as a message-based transport that knows message lengths, lets receivers pace additional packets with grants, and uses switch priority queues to favor short exchanges. In one benchmark described in the talkA benchmark is a defined test or collection of tasks used to compare a system's performance under specified conditions., Homa's 99th-percentile latency for short messages is reported at less than 100 microseconds, versus more than one millisecond for TCP, roughly a 13-fold difference. That is the speaker's sample benchmarkEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria., not an independently verified general performance guarantee.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows10
    • Coding & tools0
    • Models & learning0
    • Infrastructure & compute100
    • Research & evaluation0
    • Safety & security0
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    John Ousterhout in a dark shirt on a black background beside the blue-and-white headline HOMA FOR AI CLUSTERS. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 17 September 2026 and duration 18m 48s.

    John Ousterhout argues that Homa can cut the tail latency of short AI-cluster messages by replacing sender-led congestion control with message-aware receiver scheduling.