The speaker says GPT-6's scores changed little on a set of financial evaluations, and investigation revealed that many parts of those evaluations were flawed.

“Something like this happened with GPT-6 where we, we ran GPT-6 on a set of, um, financial evals, and the numbers didn't really change much. We're like, "Oh," like, "what's going on here? Let's take a look at this eval." We, we did a deep dive into these different evals, and lots of things were broken.”