The speaker thinks reinforcement learning may improve models through discrete, task-level capability jumps that appear smoother when averaged across tasks and combined with horizon generalization.

“bro like there is this very slow outer loop as baron mentioned of you know we will train a model and then aurella and then like the next kind of model iteration of training we will dump a bunch of these synthetic reasoning traces into the mid training data like we're kind of hitting all these quantum for all these different tasks and like on an individual task level it may look like a phase transition and like you're suddenly going from like a 0.5 percent pass rate to a 90 percent pass rate on like a particular like finance task or excel task or whatever but your average”

“average all these things together and plus the horizon generalization you kind of would go wow we've got like qualitatively better models”