The speaker previously favored building a better evaluation harness and taking advantage of general model improvements over investing heavily in complex evaluations, because he saw substantial gains in that approach.

“So it was like, do we wanna spend a lot of our time doing really complex evals, or do we just, like, wanna build? 'Cause there was so much to be gained from building a better harness and, like, just sort of leaning into everything's gonna improve, and everything kind of just improved.”