Reinforcement learning has enabled horizon generalization, in which models use more tokens over longer periods while continuing to make progress on tasks.

“and I don't think we necessarily got this like horizontal generalization like just training on math doesn't necessarily make you the greatest coder like you do have to do RL on on code environments I think what we did get though is like horizon generalization like the models just learned how to use more tokens for for longer and still make progress on some sort of task and so like you can train on environments where they get longer and longer and longer and then put them into a completely new environment and yes like”