During inference, autoregressive models generate tokens sequentially from left to right, so a token cannot be generated until all preceding tokens have been generated.

“Auto-regressive models are still sequential.”

“The computation is one left to right one token at a time.”

“You cannot generate the 10th token until you've generated everything that comes before it.”