If compute allows sampling a large enough group that multiple correct answers occur with non-negligible probability, reinforcement learning will upweight those correct answers.

“If you have a large enough amount of compute to sample a large enough group size such that your probability of getting a bunch of correct answers is past some not insignificant probability,”

“then it will be upweighted. And to Vernon's point,”