The speaker says next-token prediction proved effective despite prior arguments that a better learning objective was necessary.

“And like you can make all sorts of arguments for this and you could say, oh, humans probably don't learn how to like we don't learn how to model everything in our environment. We can't most like people can't create a photorealistic reproduction of some kind of scene they've looked at.”

“um there must be we must need a better objective but then it turned out that it just worked anyway”