No small, universal set of evaluations can answer every question needed to improve a system, so teams generally need a broad evaluation suite.

“[laughs] Um, it's just, uh, there's not, like, one perfect, like, couple sets of evals that you could put together that are gonna get you all the answers that you need to make your system better.”

“We found that people end up needing, like, a relatively large suite of these things, and then it's like you can have your agents work with you to evolve your eval suite over time as well.”