The speaker thinks exposure to algorithmic patterns in internet data develops neural-network circuits and algorithms that enable in-context learning and related capabilities.

“By observing the algorithmic patterns in the internet,”

“it actually kind of like boots up all these like little circuits and algorithms inside the neural net to do things like in context learning and all this kind of stuff. And actually, you don't actually need or want the knowledge. I actually think that's...”