causal claim
Updating model weights through gradients is highly sample-inefficient compared with learning in context.
Dwarkesh Patel · The next big breakthrough will be AIs learning on the job · 26 Jun 2026
In their words
The transcript segments this claim was taken from
“But the moment you move into the weights,”
“you have to give up on in-context learning sample efficiency.”
“Because gradient updates are super sample inefficient,”9:38Report a misattribution
0.95how strongly it was put, not whether it is true
0.95importance to the conversation
0.99confidence the claim was read correctly