Instrumental convergence proposes that AI systems with different goals will tend to seek power, greater intelligence, and physical or computational resources, while preventing barriers to their goals.

“The idea there is whatever goal it is that the AI has when it, you know, achieves this sort of basic level of intelligence,”

“whatever that goal is, that goal will be served better by having more intelligence.”

“And that intelligence in Yudkowsky's worldview is something that you can increase by, you know, throwing more computing power and memory and whatnot at it.”

“and the, the idea there is no matter what your goal is,”

“it could be something, you know, in the, in the famous thought experiment from Bostrom, it could be something as silly as, as,”

“you know, creating more paper clips,”

“keeping people in,”

“you know, uh, engaged in chatbot conversations,”

“or it could be, you know, making as much money as you can on the.”

“And on the stock market, no matter what your goal is,”

“the thought with this notion of instrumental conversions is.”

“All of those goals will lead you towards seeking more power,”

“more intelligence, more physical and computational resources in order to better achieve those goals and prevent barriers from being placed in the way of achieving those goals.”