June 29 – July 2, 2026 · San Francisco, CA · imported from ai.engineer's public schedule feed

AI Engineer World's Fair 2026 — unofficial import demo

Unofficial demo. This programme was imported from the AI Engineer World's Fair's own public schedule feed to show vibeboard at real conference scale. Not affiliated with, or endorsed by, the organisers.

All sessions
InferenceSession

Operating Distributed Inference Systems at Scale

Nishant Gupta, Naman Ahuja

When
Thursday, July 210:45 AM – 11:05 AM · 20 min
Where
Track 9San Francisco, CA · imported from ai.engineer's public schedule feed
Google Calendar

About this session

Inference has rapidly become one of the most important infrastructure problems in modern computing. As AI systems evolve into autonomous agents with persistent memory, tool usage, and multi-step reasoning, traditional inference architectures struggle under growing demands for latency, throughput, cost efficiency, and reliability. In this talk, I’ll share lessons from building large-scale elastic compute and AI infrastructure systems powering production workloads. We’ll explore the modern inference stack and the architectural patterns emerging to support next-generation agentic AI systems. Topics include distributed inference architectures for large-scale AI systems, GPU scheduling and elastic compute for inference workloads, multi-tenant inference infrastructure, caching, batching, latency optimization strategies, reliability and fault isolation for inference systems, observability and control loops for AI serving platforms, balancing cost, throughput, and user experience, and why inference is becoming an infrastructure orchestration problem. Attendees will gain practical insights into designing scalable, resilient, and cost-efficient inference platforms for modern AI workloads.

Speakers (2)

Nishant Gupta
Nishant Gupta

Software Engineer, Tech Lead, Meta

I am a Staff Software Engineer and Researcher at Meta, specializing in large-scale distributed systems, AI infrastructure, and operational resilience. Within Meta Superintelligence Labs, I build agentic infrastructure that enables AI systems to operate reliably in production through evaluation, auditing, safety controls, feedback loops, and human oversight. I previously led the development of Meta’s next-generation elastic compute infrastructure, managing roughly 30% of fleet capacity across tens of millions of servers in 20+ geo-distributed datacenters, delivering billions of dollars in infrastructure savings while shaping multi-year strategy with executive leadership. My research focuses on resource optimization, reliability, and safe AI deployment at scale. I designed and deployed Dynamic Idle Resource Leasing, a production system that safely oversubscribes datacenter capacity while preserving strict reliability guarantees. I have authored research papers with 90+ citations. I am passionate about building scalable, fault-tolerant systems and translating cutting-edge research into real-world infrastructure that delivers measurable impact.

Naman Ahuja
Naman Ahuja

Senior Software Engineer, Meta

Software Engineer at Meta working at the intersection of distributed systems, AI infrastructure, and hardware enablement. He focuses on adopting new hardware platforms across Meta’s datacenter fleet to support AGI-scale workloads and production AI systems. His work spans capacity management, autoscaling, reliability engineering, and datacenter-scale resource optimization. He has led initiatives to integrate cutting-edge hardware, improve compute utilization, and build reliable platforms for large-scale AI workloads and agentic workflows.

More in Inference

For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpoints

No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.