After LLM pretraining encountered diminishing returns, reinforcement learning helped extend progress, which then encountered another diminishing-returns curve.

“with that paradigm where you know it's basically just like self-attention rl scaling up rl environments the i guess like if you think about what happened with moore's law right like we had this very like nice straight line and that held for a really really long time but there were so many like discrete like discontinuities and innovations that had to happen to keep that scaling law going and the same thing has kind of happened with llms like we had this pre-training like scaling law and then that was kind of like you know hitting the dimension”

“diminishing returns and then we came up with like RL and solved that and then we got this new like you know diminishing returns curve to hit that made it keep looking like a straight line going up and so like if it requires another one of those discontinuities to solve like I'm not sure that like the current method of like training elements with these RL environments even like RSI targeted RL environments would be able to discover that discontinuity and if not like we're probably going to hit this like asymptotic like curve where like”