The speaker says AI research and development can combine GPT-2-scale full pretraining with small fine-tuning, post-training, or mid-training runs on GPT-6 and a limited number of frontier-scale experiments.

“And another another thing that's really important is you don't just do GPT-2 sized runs,”

“you also do small like fine tuning runs on GPT-6 or like you as in like you have GPT-2 and you can do full pre trains of GPT-2 and then you can do like small post training or mid training or whatever runs on GPT-6 and then you can do a small number of experiments that are actually”

“or actually like at frontier scale but you do a bit of online training or something”