The classic backpropagation paper argued that backpropagation produced sensible representations and supported that claim by demonstrating their interpretability.

“learning representations by backpropagating error signals right the sort of the classic backprop paper actually like most of the figures in that are them saying look the model learned sensible representations from our backprop procedure and sort of have validated it by showing that it's interpretable i think that modularity is the end point you want but you don't start with modularity um”