After a period of growth, model sizes have recently decreased somewhat.

“so it's really interesting in the history of the field because at one point everything was very um scaling pilled in terms of like oh we're going to make much bigger models trillions of parameter models and actually what the models have done in size is they've gone up and now they actually kind of like”

“actually even come down”