Generative models identify structure in data by learning a compression scheme; this applies to autoregressive models and, to some extent, diffusion models.

“Yeah, it's a good question. And fundamentally, if you think about whenever you train a generative model,”

“what you're doing is whether it's an autoregressive model or to some extent even a diffusion model is.”

“you are trying to identify structure in the data by essentially building a compression scheme.”