Compared with a Q learner, a policy gradient method can represent uncertainty about alternative actions and their expected outcomes, potentially enabling exploration of an action that is slightly worse initially but better in the long run.

“I want some breath to do better exploration. And that's the big advantage of let's say a policy gradient method over a Q learner. Because a Q learner only tells you kind of what to do. It doesn't tell you like if you do the different thing, I expect you to be worse by this and that amount. And maybe like if you had this, you would be able to go and say well for technical exploration, I choose the one which is might be slightly worse, but ends up being better in the long run.”