The speaker says current methods can perform strong, ongoing mid-training on a pretrained base and add reinforcement learning from later training checkpoints.

“the question yeah i think we have pushback like how much from scratch we need to do like it is definitely possible now to take like the pre-trained base and like do very good mid training on top of that like kind of continuously plus some rl from like different checkpoints that are later on in the training and like that's looking more like a genuine but certainly not the case of like you know take the most recent model apply a couple of very small updates and like iteratively like”