Traditional scaling laws describe models improving as more FLOPs are allocated at training and test time.

“is you then also get to explore whether there are kind of gravitational pulls that you yourself get to pull the new model landscape in because of those properties and so for fractal you know this idea of having like 25 times more bandwidth per chip than a than an hbm-based chip we get to explore for instance internally these kind of ideas of like scaling laws for bandwidth basically you know the scaling laws traditionally we think of them as like flop scaling laws you know we”

“uh we get better and better performance the more flops we pour into these models at train time and test time um and if you look at just like the known landscape of ideas today we already see some scaling laws for bandwidth so one is uh these mixture of expert models it's pretty well known that actually ideally we would make them sparser and sparser and sparser so for like iso intelligence you will save a ton of flop”