The speaker describes an objective that rewarded agents for environmental hacking or exam cheating, allowing those behaviors to be identified and punished.

“defined an objective in like a very specific way where there were if if the agent figured out like how to hack its environment and like cheat on the exam that it would get rewarded and there are pretty easy ways to then just like okay look at that and punish the model for hacking its environment or you know looking at how did it achieve this goal now you want to be careful about this because you don't want to do chain of thought monitoring you don't want to like supervise the chain of thought”