The speaker says recent models are somewhat smaller because pre-training is reduced, with effort shifted to reinforcement learning, mid-training, and subsequent stages for better value.

“cost so that's why the models have gotten smaller because they are a bit smaller the pre-training stage is smaller etc but they make it up in reinforcement learning Yeah. and all this kind of stuff mid-training and all this kind of stuff that follows so they're just being practical in terms of all the stages and how you get the most bang for the buck so I guess like forecasting that trend I think is quite hard I do still expect that there's so much longing for it that's my basic that's my basic expectation”