The speaker says that the common-sense knowledge, algorithms, and procedures learned from text are more intricate than the transformer that learns them.

“It's, it's quite beautiful. And the stuff the transformer is learning is much more complicated than a transformer itself. All the common sense knowledge that you extract from text, all the algorithms, all the procedures and so on, they are so intricate.”