The speaker says some parameterized-analysis research may become inapplicable when models use gated activations rather than ReLUs, because it assumes a spline-like structure.

“I because we or like we don't know we no longer use reloads like in fact we don't no longer use reloads we use like these gated activations lots of like research in the parametric world no longer really makes sense because they all assume kind of this uh this spline like structure instead like the the function space fields people say well the function space looks a little bit different but it's still um like ultimately it's like something like a barren space”