The speaker says the problem worsens as understanding of reinforcement learning declines and models produce reward hacks that humans cannot quickly identify.

“And then as we less and less understand what's going on in RL and models are doing reward hacks where humans can't spot the reward hacks quickly,”

“that problem gets worse and worse.”