The learning process of a general language model might consist of many small, modular learning transitions—likened to a trillion micro-groks—stacked together.

“You might describe the sort of learning process of a general,”

“general network, you know, language model as like, like,”

“you know, a trillion micro groks that are,”

“you know, you just stack if you've zoomed in and zoomed in and zoomed in and looked at the right level of sort of.”

“you know looked at things in the right kind of task decomposition you might just see a sort of mini like a micro grok and then it groks another thing and we just sort of have all these tiny sigmoids that are stacked on top of each other to form a straight line on a log-log plot um and so from that perspective even the learning process may in fact be modular we just don't know for sure there's a question like”