In the speaker's research setup, a weak language-model grader evaluates code tasks, and the model is informed that it will be graded.

“We have this really nice setup where there is a sort of weak language model grader and it's trying to do code tasks.”

“And so we tell it, you know, the only thing that's sort of slightly unnatural about it is we tell the model that it will be graded by the grader, but it has these tasks like it's given what would usually be an R or VR code task.”