Andrej Karpathy believes deep learning benefits from alternating on-demand learning with broad, exploratory learning.

“I had a small tweet about this actually so like learning things on demand is pretty nice learning depth wise I do feel like you need a bit of alternation of learning depth wise on demand you're trying to achieve a certain project that you're going to get a reward from and learning breath wise which is just oh let's do whatever one-on-one and here's all the things you might need which is a lot of school there's a lot of breath wise learning like oh trust me you'll need this later you know that kind of stuff like okay I trust you I'll learn it because I guess I need it but I love the kind of learning where you actually get a reward”