The speaker argues that model developers should assess model capabilities, make evaluations harder to game, and improve training to prevent evaluation failures instead of abandoning evaluations.

“And we just we just need to know how capable our models are like the answer is just like is hard in our evaluations and improve our training so this doesn't happen in evaluations rather than just not do evaluations. And similarly,”