Reinforcement learning has not necessarily yielded broad cross-domain reasoning transfer: math training alone does not necessarily make a model a strong coder, so code environments require code-specific RL.

“I think it comes down to two things. I think the first thing is that everyone was hoping that RL would like generalize this reasoning across like all these different domains.”

“and I don't think we necessarily got this like horizontal generalization like just training on math doesn't necessarily make you the greatest coder like you do have to do RL on on code environments I think what we did get though is like horizon generalization like the models just learned how to use more tokens for for longer and still make progress on some sort of task and so like you can train on environments where they get longer and longer and longer and then put them into a completely new environment and yes like”