In the infinite-width limit described by the speaker, a neural network can be represented by a density over weights, and its evolution can be modeled as a diffusion-style change in that distribution.

“What we can do is we can phrase it as what happens if instead of having fifty layer fifty input uh uh systems, we have an infrared number. If you put this into the infrared limit, you effectively now write this down as like a density. Right because effectively every single point exists, it just matters kind of how present it actually is. Kind of how relevant it becomes. And now”

“uh we have this the structure of effect to be fitting a density over uh over weights and kind of the yeah fitting a density over these weights. And that is very nicely channelized, because now we are back in like literal diffusion style problem, because now we can model the change in density over time as a change in distribution over time, which is ultimately a diffusion style problem. And that”