causal claim
Attention-generated fast weights enable sample-efficient in-context learning but have poor memory scaling.
Dwarkesh Patel · The next big breakthrough will be AIs learning on the job · 26 Jun 2026
In their words
The transcript segments this claim was taken from
“and models can do that in context, but using the fast weights that are built on the fly by attention,”
“which allows for the sample efficiency,”
“but scales very poorly in terms of memory.”12:09Report a misattribution
0.95how strongly it was put, not whether it is true
0.90importance to the conversation
0.99confidence the claim was read correctly