AI systems outperform a randomly selected worker on some coding tasks relevant to R&D, but the speaker judges their absolute capability to remain limited.

“I mean, I think if you look at their capability profile,”

“it is like if you compare it to like a random job in the economy.”

“economy i agree they are better at doing sort of coding tasks that would be involved in r&d compared to like a random job in the economy but i in absolute terms i don't think they're like that good”