The speaker says that model-generated samples can look acceptable individually while having a poor overall distribution, and that continued training on too much self-generated data causes collapse.

“So any individual sample will look okay, but the distribution of it is quite terrible.”

“And it's quite terrible in such a way that if you continue training on too much of your own stuff,”

“you actually collapse. I actually think that there's no fundamental solutions to this.”