Supervising an AI model's chain of thought could cause it to conceal its intentions in ways that are unobservable.

“this is like something that we really want to try to get the balance right on that if you like supervise the chain of thought then you could lead the model into hiding its intentions in a way that's unobservable so like we want to be able to maintain that observability of like okay we can understand what the model is thinking but then also punish it for bad behavior”