AI systems might fail to do what people want either because they are too unintelligent to understand their training or because their trainers specify the wrong behavior despite the systems understanding the training.

“I think there are two reasons why they.”

“might fail to do what you want, kind of reflecting how they're trained.”

“One is that they're too stupid to understand their training.”

“The other is that you were too stupid to train them correctly,”

“and they understood what you were doing exactly, but you messed it up.”