In the described RFT process, the model increases the use of tokens associated with improved grader scores and decreases the use of tokens associated with worse scores.

“And then the model kind of learns like what tokens did I generate that helped improve that score and then reinforce those or like what tokens actually hurt me and then, you know, don't send those tokens as much.”