Kernels written in practice are often far from their theoretical speed limit.

“in a lot of cases if it's even possible to hit this theoretical number if that makes sense like this is assuming like perfect overlapping and transfer of data and like there's maybe some bottleneck that you can't get around but often the kernels are not even close like that we write are not nearly close enough to this number to be like meaningful at all”