Anthropic needs to evaluate a new model’s capabilities before release to avoid deploying a dangerous model.

“would happen so okay why do we do it number one is like when we train a new model we need to understand its capabilities right and this relates to things like fallbacks and like classifiers and things like that where we don't want to put a you know like dangerous model out in the wild right and so we have to run a lot of evals again like we said the models are getting increasingly aware of it and so the evals have to be quite complex and you know”