Karpathy says computer-using agents are built on large language models, whose representations must first be developed through pretraining.

“so for example today people are training those computer using agents but they're doing it on top of a large language model and so you actually have to get the language model first you have to get the representations first and you have to do that by all the pre-training and all the LLM stuff”