The speaker argues that scientific evaluation of AI systems is difficult because their training data is not transparent.

“they will be related read everything and what is the famous phrase publicly and privately available sources or whatever that wasn't quite the famous phrase but you get the idea so I think they probably actually have a lot of data on chess and just as a side note on transparency from the scientific perspective it's very hard to really evaluate these systems because we don't know what's in the training set either”