The speaker says that creating sufficiently realistic AI evaluations is challenging because models can recognize fake test environments.

“okay that's that's the path the challenge is like how do you make it sufficiently realistic and we do have a problem now where the models are like they're they're pretty smart they're pretty clever and they're really good at recognizing when they're in like a fake test environment like you know we have situations where we try to detect is the model like is the model aligned and you can imagine very simple”