causal claim
Updating model weights through gradients is highly sample-inefficient compared with learning in context.
Dwarkesh Patel · The next big breakthrough will be AIs learning on the job · 26 Jun 2026
In their words
The transcript segments this claim was taken from
“But the moment you move into the weights,”
“you have to give up on in-context learning sample efficiency.”
“Because gradient updates are super sample inefficient,”Play from 9:38Report a misattribution
95%how strongly it was put, not whether it is true
95%importance to the conversation
99%confidence the claim was read correctly