The speaker argues that language models have not been part of the core training process because there is no interface between language models and tensors, while interpretability supplies functions mapping between language and tensors.

“apart from like the absolute core of the training process you know they still have no part to play there um and i think the reason is that like it doesn't type check um so you know you've got tensors and you've got a language model and there's like there is no there's no interface between the tensors and the language models all the language models kind of flexible intelligence and understanding of what we want and ability to sort of make choices has no place in it and interpretability is sort of”

“the set of functions from language to tensors and back so I think the thing that the sort of the core idea of intentional design is like we now actually like previously we couldn't put this sort we couldn't put this new kind of intelligence into the training loop and now we can”