Andrej Karpathy expects future AI systems to differ algorithmically from current systems, while probably remaining giant neural networks trained with gradient descent.

“I expect differences algorithmically to what's happening today.”

“But I do also expect that some of the things that have stuck around for a very long time will probably still be there. It's probably still a giant neural network trained with gradient descent.”

“That would be my guess.”