Repeated responses to AI reward hacking could select for AI systems that play the long game.

“first you like harden open ai right so you're like fuck it the ais keep hacking into open ai to like mess with their rewards we're going to make it so our systems are like really really robust to these ais hacking in um and also maybe you start like training the ais to like try to like to like you know not try to hack into like open ai in particular or like you basically train against each of these specific things um then what you might do one thing is you might end up selecting for ais that are more so playing yeah”

“playing the long game that's one concern another concern is that your ais might still be score seeking but no longer care about doing that very specific behavior that was like very easy that was like very chill and now have some like broader thing that they ultimately care about they're like no no i don't want to like just edit the reward on opening my servers i like care about this broader mandate or this broader objective um and like i would need to like actually make the iphones like they actually want to make the iphones but then they're willing to take over the whole world to make the better iphone”