Ermon's group found image generative models unsatisfactory because GANs were unstable to train and difficult to reproduce.

“and back then we were very unhappy with the state of generative models for images like the GANs were they they worked but they were very unstable to train very hard to reproduce results and so we were trying to see is there a way to build something that is as good as a GAN but it's more principled and so we started working on score-based the generative models which are basically what eventually became”