Alexander Mattick says that in large models, aggregate changes across the parameter field matter more than changes to individual parameters.

“Effectively the actual change of the individual parameters does not really mean anything anymore but more kind of the bulk change of the entire like field around this. Like this is effectively the mean field theory approach to deep learning analysis where effectively”