The speaker says any fixed AI evaluation may eventually become saturated.

“at least for this type of like reward seeking type behavior is to look at specifically the category of tasks that are like right at the limit of capabilities. And so any fixed eval maybe gets saturated, but the amount of misalignment right at the like frontier of capabilities of how people who are really pushing these AIs are using them is more concerning.”