Autoregressive models generate one token at a time, leaving relatively few opportunities to alter that generation process.

“you need to think about an autoregressive model it's pretty clear there's not a lot of things you can do there in terms of like okay you generate one token at a time and that's it in a diffusion based model we know that there is a lot of different”