A neural network's weights retain training knowledge as a hazy, compressed recollection of the training data.

“I mean, the way I usually put this is that anything that happens during the training of the neural network,”

“the knowledge is only kind of like a hazy recollection of what happened in training in the training time.”