The speaker says that after Radford discovered next-token prediction over a wide range of data, it took time for people to scale up because scaling laws and reliable prediction of outcomes had not yet been established.

“a very wide swath of data like and then even once radford discovered that right like it took a while before people decided to scale up because we had to come with the idea of scaling laws and the fact that like you can very reliably predict these things”