Effective RL and chain-of-thought training required detailed practical work, technical implementation, scaling, and correct hyperparameter choices.

“when it comes to like training AIs with to be good at reasoning and chain of thought and doing sort of RL and chain of thought training it looks like you probably could have done RL and chain of thought on like GPT-3 and gotten kind of interesting results on math if you had really scaled it up and done a good job but at the time there was low hanging fruit and also doing a good job with that training is like kind of like in the weeds and then all the technical implementation and scaling it up and getting the hyper parameters right and so”