The speaker says next-token prediction is an awkward format for using language models to perform many difficult tasks.

“Basically, when you're trying to solve a problem and you want to use a language model to solve it, like a very difficult task,”

“one thing that we have discovered is that next token prediction is a really,”

“really awkward form to do a lot of these tasks.”