Important algorithmic and architectural innovations are often concentrated in GPU-rich labs rather than GPU-poor academic or small-institute settings, suggesting that access to substantial hardware matters.

“Other kind of pieces of evidence are, you know, the fact that innovation in algorithms and architectures are often concentrated in GPU-rich labs and not in the GPU-poor parts of the world like academia or maybe smaller research institutes that also suggest that having a lot of hardware is very important. If you look at specific innovations that seem very important.”