Ermon says it remains an empirical question whether next-token prediction or a noising-based framework is preferable, and that current tools are insufficient to understand the question.

“Is next token prediction the right modeling framework or is it more like.”

“The noising and that's that's very much an empirical question that I think at the moment we don't have a tools even to understand”