Models will presumably be trained on a mixture of tasks, including easily verifiable tasks and tasks evaluated by an LLM or a human.

“Yeah, I mean, like presumably the models will be trained on like some combination of”

“of all of these tasks and like some will be very easily verifiable some will be like LMS judge or like just ask the human like does this look reasonable and then you will say the hope would be that like these would all generalize to like these much sort of hard sort of more vague fuzzy kind of tasks and like it probably will to some extent whether it generalizes enough that like we could the loop can become like self-sealing without humans being in the loop at all it's like unclear”