The speaker says AlphaGo uses Monte Carlo tree search (MCTS), which explores more than regular policy gradients.

“i mean so a couple of things here like first off i think that the alpha go is using mcts which obviously does like more exploration and”

“like stuff than regular policy gradients but i kind of also think that like rl doesn't necessarily like reduce the creativity and like i mean even if we i think you know this is obviously qualitative but if we look at like the you know the open air hiking phase incident like these models were coming up with like multiple zero days at a time to like break out of the sandbox and like this is clearly like some level of like move 37 creativity”