Advocates of current AI training argue that its sample-efficiency disadvantage applies only during training, whose one-time cost is amortized across billions of sessions.

“And the people who are in favor of the current training paradigm will say,”

“that might be true,”

“but this is only true during training.”

“And training is this one time cost that is amortized across billions of sessions that a model will experience.”