The speaker distinguishes benchmark saturation from complete coding automation and says estimating the time between the first system to saturate the benchmarks and the first system to automate coding is more speculative.

“something to scoff at they're not just multiple choice questions they're like the sort of tasks that would take a human like four hours to do or eight hours to do but that's not the same thing as complete automated encoding so first we extrapolate so when they saturate the benchmarks and then we try to make our guess as to what the gap is between the first system that can completely saturate these benchmarks and the first system that can actually automate the coding and that's probably the more speculative part”