The speaker argues that clustering by model features better exposes semantically relevant data patterns than clustering by embeddings, which also encode potentially irrelevant low-level details.

“and this is a good approximation to the way that this data point will push the model or we can also cluster based on based on features um and this is like much this is much better as a way of understanding you don't necessarily want to cluster based on embeddings um”

“because they are like embeddings contain all sorts of things that you don't necessarily care about like should i have a comma in the next like in the next token but we care about like the semantic stuff we don't care about the sort of low-level processing stuff a lot of the time so doing like doing this based on features rather than the sort of raw embeddings gives you a much better access to the stuff we actually care about we can kind of separate that out”