definition claim
The speaker describes pre-training as training a neural network to predict the next token using internet data.
Andrej Karpathy · Andrej Karpathy — AGI is still a decade away · 17 Oct 2025
In their words
The transcript segments this claim was taken from
“Basically, the thing that pre-training is doing,”
“so you're basically getting the next token”
“predictor on over the internet,”
“and you're training that into a neural net.”Play from 12:45Report a misattribution
95%how strongly it was put, not whether it is true
80%importance to the conversation
96%confidence the claim was read correctly