After mid-training, reinforcement learning mainly makes policy adjustments instead of learning behaviors from scratch, which means it needs fewer bits than might be naively expected.

“often and then what rl does on top of that is it does like a lot of you know essentially tweaking to the policy and so this is one of the reasons why it doesn't need like as many bits as you would naively think it doesn't have to learn all of these behaviors from scratch it needs just like a few bits from these episodes which you do get and then the other thing that i really point out in my blog is that these bits are actually extremely high signal compared to like regular like pre-training which is why you need rl at all versus just like sfting on like successful reasoning traces”