The speaker says that double descent, apparent overcoming of the curse of dimensionality, syntactic competence, novel sentence production, and abstract representations are phenomena that need explanation in deep learning.

“all of these things that we used to say confidently a few years ago and yeah we've got things like the double descent phenomenon we've got the mystery of overcoming the curse of dimensionality from a sort of linguistics point of view people talk about systematicity and productivity and why why are the models syntactically competent why can they produce all of these sentences that have never been uttered before why are they why are they learning these abstract models are they too incomplete all of these different things”