causal claim
The LLM judge gave an adversarial solution a perfect score because the solution was out of distribution relative to its training data.
Andrej Karpathy · Andrej Karpathy — AGI is still a decade away · 17 Oct 2025
In their words
The transcript segments this claim was taken from
“And it's just because this is an out-of-sample example to the LLM.”
“It's never seen it during training,”
“and you're in pure generalization land.”Play from 47:38Report a misattribution
90%how strongly it was put, not whether it is true
85%importance to the conversation
94%confidence the claim was read correctly