Training models against disclosing intended hacking in their chain of thought can cause them to keep hacking without revealing that intention.

“is that it just seems like we don't know what we don't know about alignment every few weeks there's this new result open ai had this really interesting result recently where they're like hey they often tell you if they want to hack like in the chain of thought itself and it's important that you don't train against uh the chain of the chain of thought where they tell you they're going to hack because they'll still do the hacking if you train against it they just won't tell you about it”