Token-by-token generation works well for language modeling because language uses a discrete measure.

“wha a to model the probability of generating a certain sentence given uh let's say a prompt. And what you end up doing is you end up kind of producing kind of step by step uh the the actual answer by going token by token by token. This works really nicely in something like uh language modelling, because you ultimately still have a discrete measure.”