The speaker says Goodfire, which he describes as an interpretability company in San Francisco, has research on geometric structures learned by models.

“where like if you change the input a little bit you'll teleport into another honeycomb somewhere else in the space and it's kind of anti manifold hypothesis but i spoke with goodfire last week they're this interpretability company in san francisco and they've got all of this work about the the geometry that these models learn so for example days of the week and months of the year they can take a pca projection of of the the activations and then they they they”

“they fit this kind of Bézier curve to it and they see these beautiful kind of geometric structures in the activation space and they're really kind of manifold”