In Karpathy's description of reinforcement learning, many candidate solutions are tried, and tokens throughout successful trajectories are upweighted to encourage repeating them.

“you will try lots of things in parallel first.”

“different attempts and these attempts can be complex right they can be like oh let me try this let me try that this didn't work that didn't work etc and then maybe you get an answer and now you check the back of the book and you see okay the correct answer is this and then you can see that okay this one this one and that one got the correct answer but these other 97 of them didn't so literally what reinforcement learning does is it goes to the ones that worked really well and every single thing you did along the way every single token gets upweighted of like do more of this”