The appropriate interpretation of a task score probably differs between evaluation and reinforcement learning uses.

“I think for on rubrics specifically, right, like you, um, the shape of this task can be used for evaluation. It could also be used for RL. And so how you think about the score probably differs, right?”

“To your point, uh, if the model's getting a lot of the things right, I don't wanna give it no, uh, you know, reward at all during an RL run.”