The speaker believes that, for recent model releases, more data is the main driver of real-world progress outside software engineering, while compute and algorithms are also being scaled and improved.

“Like if you think about, like, the latest release, I won't pick model families, but, like, it's really just collecting more data. I mean, obviously compute is, you know, they're scaling compute, and obviously they're scaling algorithmic enhancements and improvements, but, like, the main driving factor of progress in the real world in, in domains outside of software engineering is just, like, more data, right?”