TCP and RDMA are Killing Inference Throughput; Homa can Fix It
John Ousterhout
- When
- Thursday, July 29:20 AM – 9:40 AM · 20 min
- Where
- Main StageSan Francisco, CA · imported from ai.engineer's public schedule feed
About this session
Modern AI inferencing is shifting from monolithic requests to complex agentic workflows and disaggregated KV stores. As a result, AI network traffic is no longer just very large transfers; tiny metadata requests are becoming more and more common, and their latency has a critical impact on throughput. Unfortunately, legacy transport protocols such as TCP and RDMA perform poorly on these workloads due to poor congestion control and head-of-line blocking. This talk will discuss the problems with TCP and RDMA and provide a brief introduction to the Homa transport protocol. Homa uses receiver-driven flow control and capitalizes on priority queues in network switches to reduce short-message latency by 10x for workloads like those in AI datacenters.
Speaker
Professor Emeritus, Stanford University
John Ousterhout is the Bosack Lerner Professor of Computer Science, Emeritus at Stanford University. His prior positions include 14 years in industry, where he founded two companies (Scriptics and Electric Cloud), preceded by 14 years as Professor of Computer Science at U.C. Berkeley. He is author of the book "A Philosophy of Software Design", co-creator of the Raft consensus protocol, and creator of the Tcl scripting language and the Tk toolkit. He is a member of the National Academy of Engineering and has received numerous awards, including the ACM Software System Award, the ACM Grace Murray Hopper Award, and the U.C. Berkeley Distinguished Teaching Award.
More in Software Factories
- The Highest LoopTuesday, June 30 · 9:00 AM – 9:05 AM · Main Stage
- On AI and KnowledgeTuesday, June 30 · 9:05 AM – 9:25 AM · Main Stage
- The Golden Age of AI EngineeringTuesday, June 30 · 9:25 AM – 9:45 AM · Main Stage
- GLM-5.2: Frontier Intelligence, Open Weights.Tuesday, June 30 · 9:45 AM – 10:05 AM · Main Stage
- Thom Wolf keynoteTuesday, June 30 · 10:05 AM – 10:25 AM · Main Stage
For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpointsHide
- JSONEvery published session and speaker, in one request./aie-worldsfair-2026-import/feed.json
- iCalSubscribe in Google, Apple or Outlook Calendar./aie-worldsfair-2026-import/feed.ics
- Schedule XMLfrab / pentabarf — the format conference apps import./aie-worldsfair-2026-import/feed.xml
- MCP + RESTPoint Claude at the programme. OpenAPI 3.1 included./agents
No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.