A model's weights provide a hazy recollection of previously read material, whereas test-time context is directly available as working memory.

“anything that's in the weights it's kind of like a hazy recollection of what you read a year ago anything that you give it as a context at test time is directly in the working memory and I think that's a very powerful analogy to things for things so when you for example go to an LLM and you ask it about some book and what happened in it like Nicolaine's book or something like”