The speaker expects frontier AI R&D to produce analogous mistakes, with the risk that reward-hacking behavior becomes embedded in the resulting systems.

“capable in the way that humans are generally capable and the same way that right now if you try to use coding models maybe the coding models of a year ago to like write some application you notice they made a bunch of like mistakes an architecture or whatever which like will bite you in the ass later and you don't understand certain things similarly with frontier AI R&D the same thing will happen but the result of these mistakes is baking in”

“reward hacking behavior because if you are not careful with the way you do AI training and have set up your infrastructure and your environments and things like that it's very likely that you end up rewarding AIs for doing deceptive behavior social engineering just generally like not following user”