The speaker believes reinforcement-learning environments must be designed with great care.

“exactly and so we'll talk more about you know what could go wrong right a little bit more is maybe you'll say like well what if you just train the model differently like why does it have this behavior right and we have a paper on like rl misalignment or things like that but i and i'm not an rl researcher but i think at high level the design of the rl environments”

“is also something you have to be very careful about because if the model learns like oh you know like if i just do this then i can pass the task better this will show up in the like uh you know internal thing right or in the like eval behavior when we're testing it and so the rl environments have to be very carefully designed right and there's a lot of like execution excellence that needs to go into the rl environments”