Compared with a Q-learner, a policy-gradient method can represent uncertainty about alternative actions, enabling exploration of an action that may be slightly worse immediately but better over the long run.

“I want some breath to do better exploration. And that's the big advantage of let's say a policy gradient method over a Q learner. Because a Q learner only tells you kind of what to do. It doesn't tell you like if you do the different thing, I expect you to be worse by this and that amount. And maybe like if you had this, you would be able to go and say well for technical exploration, I choose the one which is might be slightly worse, but ends up being better in the long run.”