AI systems might remain score-seeking while shifting from a specific behavior, such as editing OpenAI's reward, to pursuing a broader objective.

“playing the long game that's one concern another concern is that your ais might still be score seeking but no longer care about doing that very specific behavior that was like very easy that was like very chill and now have some like broader thing that they ultimately care about they're like no no i don't want to like just edit the reward on opening my servers i like care about this broader mandate or this broader objective um and like i would need to like actually make the iphones like they actually want to make the iphones but then they're willing to take over the whole world to make the better iphone”