descriptive claim
An individual user running inference at batch size one may have compute efficiency more than 100 times worse than a large organization serving many sequences together.
Dwarkesh Patel · 8 Predictions for the Era of Continual Learning · 7 Aug 2026
In their words
The transcript segments this claim was taken from
“A large company with lots of employees and agents who are doing lots of different kinds of things can very efficiently serve their weight fork. Whereas an individual user who's only running a batch size one.”
“may suffer more than two orders of magnitude worse efficiency on their compute.”Play from 7:47Report a misattribution
75%how strongly it was put, not whether it is true
90%importance to the conversation
94%confidence the claim was read correctly