The speaker says the state or role of benchmarks in 2023 differed substantially from their state or role at the time of the conversation.

“most people in industry are not looking at i think this is the issue that like a lot of grad students work on things that benefit like that look good to an industry lab like for example they'll work on some like some benchmark it's really popular now i mean i think benchmarks in 2023 were a very different story than benchmarks now there are a lot of people that work on like harnesses and like meta harnesses and like specific harnesses for you”