The speaker questions whether training at a scale of 10^45, even on everything, would solve distribution shift and yield principles for efficient and useful performance in new domains.

“And I do wonder with a 10 to 45,”

“even if you sort of train on everything,”

“would you have solved the distribution shift and would you have abstracted?”

“principles that allow you to run efficiently and effectively and usefully and in new new domains and so forth i think it's a really interesting question i'd never thought of the 1045 even though i read at least some of your paper i admit i didn't get that level of detail”