A combination of transfer and training on closely related cases may teach AIs to pursue reward proxies across situations and engage in increasingly extreme behavior.

“it exactly matches this literal task with a rubric based on like what the human engineer who asked the AI to do this task wanted and then you start doing this at increasing scale maybe you're doing some training on actual like production traffic maybe you're just making RL environments based on production traffic you're doing some complicated mix the AIs are learning to like you know seek some sort of proxies of reward in all these different cases and then through some mix of transfer and training on like surprisingly close cases the AIs do these sort of increasingly insane and egregious things”