descriptive claim
Andrej Karpathy says PyTorch’s DistributedDataParallel container synchronizes gradients across GPUs during the backward pass.
Andrej Karpathy · Andrej Karpathy — AGI is still a decade away · 17 Oct 2025
In their words
The transcript segments this claim was taken from
“The way to synchronize gradients between them is to use a distributed data parallel container of PyTorch, which automatically does all the,”
“as you're doing the backward,”
“it will start communicating and synchronizing gradients.”Play from 31:54Report a misattribution
95%how strongly it was put, not whether it is true
60%importance to the conversation
97%confidence the claim was read correctly