Toward the end of 2023 and especially in 2024, Fractile became concerned about the scalability of its SRAM-based chip architecture.

“and let's say the weights of your model or the kv cache of this model as you're rolling it out um and that's something which then then can drive you to thousands of tokens per second on these language models um but i think one of the things that we started to worry about in kind of towards the end of 2023 and certainly in 2024 was the scalability of this approach um and you know i think there are two things that”