
The Race to Production-Grade Diffusion LLMs with Stefano Ermon
Episode 764
Today, we're joined by Stefano Ermon, associate professor at Stanford University and CEO of Inception Labs to discuss diffusion language models. We dig into how diffusion approaches—traditionally used for images—are being adapted for text and code generation, the technical challenges of applying continuous methods to discrete token spaces, and how diffusion models compare to traditional autoregressive LLMs. Stefano introduces Mercury 2, a commercial-scale diffusion LLM that can generate multiple tokens simultaneously and achieve inference speeds 5-10x faster than small frontier models, paving the way for latency-sensitive applications like voice interactions and fast agentic loops. We also cover the open research challenges in diffusion LLM training, serving infrastructure requirements, and post-training for diffusion-based systems. Finally, Stefano shares his perspective on whether diffusion models can rival or surpass autoregressive LLMs at scale, the advantages for highly controllable generation, and what the future of multimodal diffusion models might look like.
Publisher
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Link
twimlai.com/podcast/twimlai/race-production-grade-diffusion-llms
Published
March 26, 2026
Duration
1h 3m
Format
Podcast
Category
Issues · Tech News
Source
RSS feed
Host
Sam Charrington
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
@twimlai
#production #gradediffusionllms #stefanoermon
