One possible approach is for an AI company to convert tasks on which its AI performs poorly into reinforcement-learning environments, using rubrics based on the human requester's intent.

“But like you might imagine,”

“for example, that within the AI company,”

“they use the AI to do work.”

“And then they're like, huh,”

“the AI did a really bad job on this task.”

“Maybe we should take this task and turn it into an RL environment.”

“it exactly matches this literal task with a rubric based on like what the human engineer who asked the AI to do this task wanted and then you start doing this at increasing scale maybe you're doing some training on actual like production traffic maybe you're just making RL environments based on production traffic you're doing some complicated mix the AIs are learning to like you know seek some sort of proxies of reward in all these different cases and then through some mix of transfer and training on like surprisingly close cases the AIs do these sort of increasingly insane and egregious things”