The speaker says their benchmark team prioritizes problems humans can solve rather than maximally difficult problems that nobody understands.

“But one thing I do want to emphasize here is one of the things that sets us apart from other benchmark creators is we don't aim for PhD plus plus problems like the hardest possible thing that nobody understands.”

“Our anchor and our constraint,”

“which is actually really freeing, is can humans do this, right?”