normative claim
Rubric criteria should avoid being vague enough to enable agent reward hacking or so specific that they rule out good answers.
Unresolved speaker · What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team · 9 Nov 2026
In their words
The transcript segments this claim was taken from
“I have to make sure that no individual criteria I've written in that rubric is, uh, uh, too vague, such that an agent could reward hack it or too specific, such that it would, you know, exclude a good answer.”10:38Report a misattribution
0.95how strongly it was put, not whether it is true
0.80importance to the conversation
0.98confidence the claim was read correctly