The speaker estimates that a B300 used for inference can generate about $40 per hour at full utilization and about $25 per hour at roughly 50% utilization, for inference on any model.

“The place where I'm like, oh man, no shit, um is that um right now, if you took a B_ three hundred and you trying to run inference on it, um you can make forty dollars an hour or so um selling that”

“inference.”

“Any model basically. Um and”

“so that's if you use it at a hundred percent. And maybe if you only have like fifty percent utilization or something, so like twenty five dollars an hour.”