The speaker predicts that the Apollo paper will probably appear in the training data for the next model.

“um about this it probably has a bunch of specific examples you know this apollo this apollo paper will probably be in the training data for the next model um and so they're going to be like um we've already told them about the existence of this stuff like right from the start and so it shouldn't be too surprising um that they like this is sort of this is at least implicitly on the list of possibilities”