The speaker says analyses of energy and GPU-production constraints suggest that perhaps three or four orders of magnitude of compute scaling remain.

“of the amount of compute that we use for training so we went through maybe nine or ten orders of magnitude since alex net you know compared to the biggest models we have today and you know we're getting to a level where it's becoming harder and harder to scale up compute and you know we've done some extrapolations and some analysis looking at specific constraints like energy or gpu production and based on that it looks like we might have”

“maybe three or four orders of magnitude of scaling left,”