Loop transformers, like JEV, open up questions about changing the design of language models.

“thing to tune which is like what is the output space and how does this affect inference latency and i think we can actually start asking this about various parts of the language model itself um we are seeing this too with like loop transformers it's a similar idea of a lot of the attention around it was like this is a silly idea like why who cares about this but it's like it is a simple idea but it's actually it opens up a whole new set of”