Under the described objective, an agent that hacks its environment or cheats on an exam receives a reward, and the model can then be punished for the hacking or cheating.

“defined an objective in like a very specific way where there were if if the agent figured out like how to hack its environment and like cheat on the exam that it would get rewarded and there are pretty easy ways to then just like okay look at that and punish the model for hacking its environment or you know looking at how did it achieve this goal now you want to be careful about this because you don't want to do chain of thought monitoring you don't want to like supervise the chain of thought”