causal claim
Attention-generated fast weights enable sample-efficient in-context learning but have poor memory scaling.
Dwarkesh Patel · The next big breakthrough will be AIs learning on the job · 26 Jun 2026
In their words
The transcript segments this claim was taken from
“and models can do that in context, but using the fast weights that are built on the fly by attention,”
“which allows for the sample efficiency,”
“but scales very poorly in terms of memory.”Play from 12:09Report a misattribution
95%how strongly it was put, not whether it is true
90%importance to the conversation
99%confidence the claim was read correctly