After applying diffusion models to images, Stefano Ermon explored using them for text, code, and other discrete objects as a potentially more parallel alternative to autoregressive generation, with built-in error correction.

“And so since we were able to get them to work on images,”

“I started thinking about how do we get diffusion models to work on text or code generation and discrete objects.”

“Like, is there a way to move beyond autoregressive models?”

“goes to something that it's more parallel, more with built-in error correction.”