Optimists argue that scaling training further can overcome current training’s fundamental deficits, including data inefficiency and the lack of internal learning.

“And people who are optimistic about this vision will say that all these things that we talk about as the fundamental deficits in the current training paradigm,”

“for example, the data inefficiency of these models or the fact that they lack internal learning,”

“these things can just be steamrolled if we just scale training more.”