Karpathy expects NVIDIA to continue gradually tuning its hardware, including Tensor Cores.

“our hardware um our old kernels um older kernels for running the hardware maximizing what you get with the hardware you know so nvidia slowly tuning the actual hardware itself tensor course and so on all that needs to happen and will continue to happen uh all the kernels will get better and utilize the chip to the max extent all the algorithms will probably improve over optimization architecture and um just all the modeling components of how everything is done and what the algorithms are that we're even training with so i do i do kind of expect like a”