AI developers will likely combine human feedback and practice environments, addressing the most apparent weakness in each successive iteration.

“People will in practice do some combination of those two things and just each iteration like patch whatever seems to be most broken in the last iteration. So like researchers will be using the AIs a lot and will notice that they have some consistent weaknesses and then those things will either be patched by like collecting human feedback or like creating environment.”