descriptive claim
Andrej Karpathy says PyTorch’s DistributedDataParallel container synchronizes gradients across GPUs during the backward pass.
Andrej Karpathy · Andrej Karpathy — AGI is still a decade away · 17 Oct 2025
In their words
The transcript segments this claim was taken from
“The way to synchronize gradients between them is to use a distributed data parallel container of PyTorch, which automatically does all the,”
“as you're doing the backward,”
“it will start communicating and synchronizing gradients.”31:54Report a misattribution
0.95how strongly it was put, not whether it is true
0.60importance to the conversation
0.97confidence the claim was read correctly