In the early days of generative modeling, producing grainy digit images from MNIST counted as a major success.

“We were still like building little generative models over MNIST and it was like a big success if you could generate these grainy images of digits and, you know, it was even hard to publish papers back then on that topic and you had to kind of like justify training a generative model as a way to learn features from unlabeled data that then could maybe help you do better at supervised learning because”