The speaker reports that AI performance was poor on a chess task involving a position with three white queens and an immediate win for black.

“which was like I had some variation on chess like give me a board state where there are three queens on white side but black can immediately win and performance was not great so like you you take knowledge that for an expert would be out of what they've specifically practiced but they understand the domain well enough and experts will have no trouble at all and these systems still do”