The speaker argues that external memory and a modestly longer context window would still be insufficient for some tasks, which require more than the model's memory context.

“um yeah i think there's so many things in the world which like even if you have some sort of memory system external to the model and even if like context length grows a little bit like there are just fundamentally things like even if you could research the information or write notes yourself like you'd need more than the memory context field to do”