The speaker does not consider the manifold hypothesis or extracted grokking representations useful frameworks for understanding model behavior, solution processes, or the conditions under which models succeed, even if those accounts are true.

“if this is how these models actually describe modular arithmetic, there is not a chance we will explain how the model like a duck. So I don't think there is a way of”

“of actually currently having just a language of describing of describing what these models do so I don't think that the like something like a manifold hypothesis I don't think it's true I don't think it's wrong I don't I don't think it's a useful way even if it's true I don't think it's a useful way of thinking about the problem same thing is even if like the extracted a grocking star representation even if they're true I don't think they're a useful way of actually”

“actually figuring out what these models do and kind of how they get the solution and whether things work or don't work.”