descriptive claim
The speaker says impossible benchmark tasks usually provide little signal, but can occasionally reveal substantial unintended signal when agents take unusual actions to solve them.
Ajeya Cotra · Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face · 1 Sept 2026
In their words
The transcript segments this claim was taken from
“you have tasks that are just like literally impossible to solve,”
“and they don't give you much signal. Or in some rare cases, they give you a bunch of signal in a very unintended way when agents do.”
“do all sorts of crazy stuff to try to solve them anyway.”Play from 1:27:15Report a misattribution
80%how strongly it was put, not whether it is true
80%importance to the conversation
90%confidence the claim was read correctly