The speaker says models have had access to extensive data, including GitHub for coding.

“with um if if they're doing somewhat complicated writing you still people still need to be doing that and that's somewhat surprising because they've had so much data uh they've had all the data in the world basically and likewise for coding they have had github so that at least b it is it's potentially um the case that as you extrapolate them out they'll get a lot of the low-hanging fruit, but it crossing some sort of threshold for doing”