Diffusion-based models allow multiple approaches to allocating training compute and improving inference-time quality, including distillation and differential-equation-solving techniques to accelerate sampling.

“you need to think about an autoregressive model it's pretty clear there's not a lot of things you can do there in terms of like okay you generate one token at a time and that's it in a diffusion based model we know that there is a lot of different”

“possibilities for training compute for quality at inference time like even if you think about image diffusion models or video diffusion models there is a lot of techniques that you can use to kind of like accelerate sampling of distillation or like fancy differential equation solving techniques that allow you to sample very very quickly from these models and so there is a lot of research on the training on the inference and then engineering like just”