The interviewer argues that a theory is needed to explain phenomena including double descent, apparent overcoming of the curse of dimensionality, systematicity, productivity, and models' ability to produce novel syntactically competent sentences.

“all of these things that we used to say confidently a few years ago and yeah we've got things like the double descent phenomenon we've got the mystery of overcoming the curse of dimensionality from a sort of linguistics point of view people talk about systematicity and productivity and why why are the models syntactically competent why can they produce all of these sentences that have never been uttered before why are they why are they learning these abstract models are they too incomplete all of these different things”

“And we need to have a theory that kind of explains this phenomena.”