The speaker says current LLMs lack an equivalent of active reflection and reconciliation during learning, and that implementing such a process remains a research problem.

“and by manipulating that information that you actually gain that knowledge and i think we have no equivalent of that again with other lumps they don't really do that but i'd love to see during pre-training some kind of a stage that thinks through the material and tries to reconcile with what it already knows and thinks through for like some amount of time and gets that to work and so there's no equivalent of any of this this is all research there's some subtle”

“Very subtle that I think are very hard to understand reasons why it's not trivial.”