A function-space perspective interprets increasing model parameters as approaching the target function space, rather than necessarily as harmful overparameterization.

“From a function point of view, you say, well, no it's not over-parameterised, I'm just going closer and closer and closer to the actual true function space I'm fitting to.”

“A a a p uh a function with just say a billion parameters is much much closer than a function with fifty parameters to being at this like infinite uh scaling domain.”