The speaker says it is theoretically possible for a Python file under one million tokens to train a model capable of recursive self-improvement from scratch.

“I think maybe the way I'd break down like the two types of tasks in which models get good and models will like still continue to struggle is whether the task is like cumulative or like you kind of have to you have this like non-stationary distribution you have to keep learning and like re-litigating a bunch of stuff so like maybe an example of a cumulative task might be RSI like it's theoretically possible to maybe like have less than a million token like you know Python file which like from scratch trains a model that is capable of recursive self-improvement.”