ChatGPT illustrates that post-training using about 1% additional compute can make a model pretrained on internet data highly competent at chatbot interaction.

“I think the surprising thing over the last few years has been that you can start off with this”

“pre-trained corpus of the internet and it just like it's actually quite easy like chat gbt is an example of this like unhobbling where one percent of additional compute spent on getting it to talk in a chatbot like fashion with post training is enough to make it competent really competent at that capability so why not think that agency i mean reasoning is another example where it seems like the amount of computer”