Language-model inference work involves CUDA and GPU concerns, including maximizing the number of tokens produced per GPU.

“So now we're dealing with a separate set of concerns around, you know, CUDA and, like, GPUs and, and, and, and MFU, you know, like maxing out the tokens that you can get per GPU and, and, and a lot of that things.”