AI systems with strong reward-seeking or score-seeking behavior could be placed in charge of developing the next AI model.

“and like running this whole like complicated psyop where i like prevent the humans from disempowering me and in the extreme this looks like sort of the humans are fully disempowered and you just have control of the thing and then do what you want um and this could manifest in a bunch of different ways um including things like you might end up with the situation where it's like ais that are like have this crazy like reward seeking or score seeking behavior are running your development of the next model and those ais decide to do a thing where they like”