According to a colleague quoted by the interviewer, RLV helps with exploration because the base language model has substantial conceptual understanding.

“the amazing things about RLV are is that it kind of it doesn't solve but it helps the exploration problem because the base LLM knows so much so it has lots of conceptual understanding and rather than just blindly searching all over the place because obviously back in the days of deep Q networks and whatnot we actually needed to guide and and put a lot of priors into the exploration process because otherwise not very many interesting things will happen and now of course with the RLV are on the language models that they have a lot”