The speaker says the traditional view was that foundation models memorize long-tail information and interpolate within that learned space, but argues that this does not describe what is happening now.

“So, you know, the story always was we had these big foundation models and they just memorize a bunch of the long tail and then we can just do interpolation or something, you know, inside that space.”

“But I don't think that's what's happening now. I think the model.”