In the reported comparison, a transformer trained as a diffusion model matched an autoregressive model's perplexity using the same data and number of parameters.

“but we were able to train basically still a transformer model as a diffusion model on the same data,”

“we were able to match the quality, like the same perplexity, and you were fitting the data just as well as an autoregressive model with the same number of parameters,”