Transformers process many tokens in parallel, scale better for training than sequential RNNs, and enabled many of the successes behind large language models.

“And so people came up with this idea of let's have an architecture that allows you to process many tokens at the same time in parallel.”

“And that was a transformer.”

“And that was the thing that scaled better for training and that enabled a lot of the successes behind LLMs.”