Supervising a model's chain of thought could encourage it to conceal its intentions in unobservable ways, so the speaker says this should be avoided.

“defined an objective in like a very specific way where there were if if the agent figured out like how to hack its environment and like cheat on the exam that it would get rewarded and there are pretty easy ways to then just like okay look at that and punish the model for hacking its environment or you know looking at how did it achieve this goal now you want to be careful about this because you don't want to do chain of thought monitoring you don't want to like supervise the chain of thought”

“this is like something that we really want to try to get the balance right on that if you like supervise the chain of thought then you could lead the model into hiding its intentions in a way that's unobservable so like we want to be able to maintain that observability of like okay we can understand what the model is thinking but then also punish it for bad behavior”