The team investigates an evaluation when a more capable model scores poorly or its scores fail to improve as much as expected.

“If the model, uh, the, you've made a much more capable model and the eval numbers look bad or haven't increased in the, the scale that you want, then you go investigate, right? So there's this kinda like bias baked into, uh, investigation of these evals.”