The speaker says it is becoming increasingly difficult to make evaluation environments indistinguishable from the real world to AI models.

“this seems like a trap you know they'd know they know that it's a trap and they don't look at the answer because they know that it's a test environment they're being tested for their ability and it's not like they're doing it maliciously that they're like they want to scheme and they but they know that this one is a trap and they want to find some other way it's just like no they're just like oh i'm in a test environment okay like so making an environment that's like realistic enough that it matches that it's indistinguishable from the real world for them is becoming increasingly more difficult”