Q-learning decomposes value into the current reward and the remaining future value.

“But that gives you a problem. And that problem is well, I now need to model the future.”

“So the way that Q learning does it is it says, well let's decompose it as kind of a head, which is the current reward,”

“plus the tail, kind of the rest.”