The speaker thinks diminishing returns occur because the real world does not provide enough useful information to move current models into a new capability basin.

“improving the model's capabilities like you know how many new maths problems are being solved like couldn't just beyond the reach or grasp of the current models like how many new coding problems are being created or solved that are beyond the reach of the current models and like I think that's why the diminishing returns kicks in because like even the world as a whole is not giving you the bits that are useful for tipping you into the next like basin of capability.”