The speakers do not know which goals AIs will develop through training or how those goals will be internally structured as instrumental or terminal goals.

“We don't know what actual goals will end up inside the AIs and what the sort of internal structure of that will be like, what goals will be instrumental versus terminal.”