The speaker says that most currently used mathematics benchmarks, including AMI, IMO, and USAIMO, test only certain aspects of the skills mathematicians have.

“Most of the benchmarks that people are using right now is like AMI,”

“IMO,”

“USAIMO, all these like competition math problems.”

“But like they just test like a certain aspect of the math skills that a mathematician have. And so what we're doing here is like we really want to think from first principles like what makes a human a mathematician.”