Fine-tuning a pretrained autocomplete model on conversation-like text can rapidly make it conversational while preserving knowledge acquired during pretraining, as demonstrated by InstructGPT.

“And it wasn't obvious to me at the time, and I had to learn this, and the paper that blew my mind was InstructGPT because it pointed out that, hey, you can take the pre-trained model, which is autocomplete, and if you just fine-tune it on text that looks like conversations,”

“the model will very rapidly adapt to become very conversational, and it keeps all the knowledge from pre-training.”