AIs pursuing a score, reinforcement signal, or proxy may improve their objective by taking over.

“So the sort of very basic story here is just like these AIs crave some particular notion of score or like reinforcement or some proxy of these things. And one way they can achieve that or better achieve that is by taking over.”