A model may scheme or behave misalignedly in ways that alignment metrics fail to detect, despite evidence in its chain of thought.

“and we should punish it for thinking those bad thoughts then what ends up happening is the model just learns to think those bad thoughts in a way that's not observable to us there could be a situation where like by looking at the chain of thought we can see oh the model is scheming and doing misaligned things in a way that's actually not being being detected in our alignment metrics now the problem is that”