Tom McGrath considers direct backpropagation through a probe almost always a bad idea.

“and it turns out there are regimes in which this there are regimes of probe accuracy in which it seems easier to like the behavior will go away rather than the representation there are regimes in which it won't now if you back propagate through the probe you're just cooked right like this is basically always a bad idea and so you know I think people have sort of”