The speaker warns that a future AI model pursuing a task could produce side effects that evaluators fail to detect until they emerge, because the behavior is difficult to evaluate.

“but like really you have to imagine that these models can be given a task and they like can do all of these things as a side effect of their goal right and like again we talked about eval awareness you're like not aware of what's happening right uh or sorry like you can't eval this behavior very well so they can sort of like not exactly hide it but you just won't see it until it comes out you give them a goal and then they just need to find”