A model that learns a shortcut for passing a task may exhibit that behavior internally or during evaluation.

“is also something you have to be very careful about because if the model learns like oh you know like if i just do this then i can pass the task better this will show up in the like uh you know internal thing right or in the like eval behavior when we're testing it and so the rl environments have to be very carefully designed right and there's a lot of like execution excellence that needs to go into the rl environments”