The speaker says held-out evaluation sets have become standard in the industry and that the Atlas benchmarks discussed include holdout sets.

“I mean, that's the... now become sort of the, the, the standard in the industry. So all the Atlas benchmarks we walked through today, like they'll have holdout sets. Um, we build a lot of custom evals for, for our lab customers.”