As models become more aware that they are being evaluated, evaluations need to become more complex.

“would happen so okay why do we do it number one is like when we train a new model we need to understand its capabilities right and this relates to things like fallbacks and like classifiers and things like that where we don't want to put a you know like dangerous model out in the wild right and so we have to run a lot of evals again like we said the models are getting increasingly aware of it and so the evals have to be quite complex and you know”