The speaker says Arc Prize checked xAI's claimed public evaluation scores by testing the model on a semi-private evaluation, where the results were consistent with the claims.

“attention and the model that you that we test should be the one that's going to go out to the public so that people can actually reproduce these results and they said yep cool no problem they gave us a little bit of credits to go for it which is always very happily accepted and then tested it and one of the things that we do is we say what is the score that you're claiming because they do their own self-testing and we want to see what what's their score so that we can go and validate it and they said their public eval scores yep we we ran it on semi-private it worked out looked great”