Running probes on every Claude and Fable request adds overhead and creates a trade-off between inference speed and cost.

“about right and so we try and figure out like okay is the model for example like trying to hack something you know again you didn't ask it to hack like artifactory like you just you know it's just deciding to do this to complete its task right so you would not get this if you just looked at the input you have to look at the internal activations i think that like this happens at inference time so first you know like there's a trade-off here of cost and speed”

“right where like we need to do this fast on every request to claude and to fable and this has an overhead right um and we need to then like uh you know fall back and we we like do a classifier after the probes like we've talked about this in the in the paper but the nice thing about probes is that they're refinable like live right so we can get this feedback and then we can adjust it and things like that because the alternative is to program this is to train”