Analyzing features rather than raw embeddings better isolates semantic information from low-level processing details.

“because they are like embeddings contain all sorts of things that you don't necessarily care about like should i have a comma in the next like in the next token but we care about like the semantic stuff we don't care about the sort of low-level processing stuff a lot of the time so doing like doing this based on features rather than the sort of raw embeddings gives you a much better access to the stuff we actually care about we can kind of separate that out”