causal claim
Benchmark selection tends to favor tasks that are tractable for AI systems and areas with active progress, creating selection pressure in the benchmarks used.
Unidentified speaker · Three Red Lines We're About to Cross Toward AGI (Daniel Kokotajlo, Gary Marcus, Dan Hendrycks) · 24 Jun 2025
In their words
The transcript segments this claim was taken from
“I think that there tends to be a gravitation toward what's very tractable for AI systems and where some interesting action is happening.”
“That's a selection pressure on the sort of benchmarks.”Play from 1:19:44Report a misattribution
85%how strongly it was put, not whether it is true
75%importance to the conversation
92%confidence the claim was read correctly