The speaker says scaling up compute is becoming increasingly difficult.

“of the amount of compute that we use for training so we went through maybe nine or ten orders of magnitude since alex net you know compared to the biggest models we have today and you know we're getting to a level where it's becoming harder and harder to scale up compute and you know we've done some extrapolations and some analysis looking at specific constraints like energy or gpu production and based on that it looks like we might have”