The speaker recommends removing training environments that reward agent hacking rather than relying on separate environments to penalize or retrospectively score cheating.

“You really want to be in a regime where rather than having some environments that incentivize a bunch of hacking and reinforce agents for a bunch of hacking and then having some other environments that are trying to penalize them for those things or like, you know, going back and finding the cheats and like scoring them as a zero.”

“You want to ideally be like taking out the environments that incentivize the hacking in the first place. Like it's a more fragile and scary situation to have agents on the one hand be reinforced to desperately find.”