The speaker says continuous mid-training and checkpoint-based RL do not mean that repeatedly making small updates to the latest model will prevent information loss.

“the question yeah i think we have pushback like how much from scratch we need to do like it is definitely possible now to take like the pre-trained base and like do very good mid training on top of that like kind of continuously plus some rl from like different checkpoints that are later on in the training and like that's looking more like a genuine but certainly not the case of like you know take the most recent model apply a couple of very small updates and like iteratively like”

“you like never lose”

“it but”