What is an Inference Engine, Anyway?

Workshops Day 1Workshop

Charles Frye

When: Monday, June 2911:05 AM – 12:05 PM · 60 min
Where: Track 8San Francisco, CA · imported from ai.engineer's public schedule feed

About this session

To run state-of-the-art inference yourself, you must master the inference engine: vLLM, SGLang, TRT-LLM, or your own jawn. The inference engine manages the lifecycle of an inference request, from input to output. In this workshop, we'll examine the architecture of modern high performance inference engines, the key techniques that inference engines need to deliver that performance, and the traces and metrics that inference engines emit.

Speaker

Charles Frye

Member of Technical Staff, Modal

Charles Frye builds and teaches people to build AI applications. After publishing research in psychopharmacology and neurobiology, he got his Ph.D. at the University of California, Berkeley, for dissertation work on neural network optimization. He has taught thousands the entire stack of AI application development -- from linear algebra fundamentals and GPU arcana to building defensible businesses -- through work at Weights and Biases, Full Stack Deep Learning, and Modal.

More in Workshops Day 1

For developers: this programme is open data — JSON, iCal, schedule XML and an MCP endpoint.Show endpoints

No key, no signup, CORS open. Everything here is generated from the same data the organisers edit.