Tom McGrath describes the sparse autoencoder's core inductive bias as representing features along lines or positive rays from the origin.

“which might seem like a really wacky thing to do, because what we're asking is like, how much structure, which is not in the form of a line, is that, and the core inductive bias of the SAE is that”

“is that things like lie on on lines you know everything is everything is a ray out from the origin or you know sort of positive ray um and so that might seem like a really weird thing to do but uh say why it makes sense um and the idea is that they're let's say that i let's just say for the sake of argument i have a feature and it just lies on an arc”