The speaker says reinforcement-learning-trained language models understand things differently from humans but address the exploration problem by learning how things fit together and exploring novel trajectories.

“problem in reinforcement learning that you know when it found move 37 it was kind of innovative but not creative it was creative to us because we understood the creative space we understood how it all hung together and now with these RL trained language models they don't understand it quite in the way that we do but they solve the exploration problem by understanding how things fit together so they can just explore novel trajectories they have a lot of base knowledge they clearly do things”