Even if the relevant AI capabilities had existed in 2017, researchers would still have needed to make the subsequent discoveries, including the usefulness of language models and fine-tuning them with reinforcement learning.

“great question we still have to like stumble through all the discoveries that we've made since 2017 we still have to like figure out that language models are a thing we still have to like figure out that you can fine-tune them with rl like like so”