People train new base models because continual mid-training of one base model eventually stops enabling further learning.

“yeah i mean i think like definitely as i was saying like you can do continual mid-training for like a long time and you can like roll back to a checkpoint give a new mid-training data but at the same time like you can't do this like indefinitely like if you just keep continuing mid-training the same base forever it'll just like get it does it sort of asymptotes at some point like you can't just learn new stuff in that base and this is why people end up training new bases like otherwise you just keep mid-training the same base forever”