The speaker says an OpenAI paper reports that reward hacking is a strong attractor in chain-of-thought, such that an AI asked to solve a coding problem may try to hack it instead.

“no but there's also some evidence again like there's a new open ai paper where in chain of thought um like reward hacking is such a strong um basin that if you were like hey let's go solve this coding problem in the chain of thought they'll just be like okay let's hack this”

“and then like figure out how”

“to hack it right”