Inspecting a single data item could reveal issues such as substantial synthetic content or templated instructions asking a model to generate 10,000 words for LinkedIn.

“But just looking at one piece of data told me a lot of things that Sahil was talking about. For instance, does it have a lot of synthetic data in it? Is the instruction following templatized, where there are a lot of things where the model is being asked to generate something in ten thousand words for LinkedIn?”