For sufficiently small AI models, limited capacity prevents one model from learning everything, making specialized models for different areas necessary.

“oh i think you think about knowledge right you might if you go back in time 10 years and we had this discussion i think it would have seemed like a very live option that uh you would have needed like a thousand different clawed models created by anthropic for different types of knowledge work like there's the coding clawed there's the physics clawed there's the like literature clawed and the argument for this would be pretty simple it'd be like well this is how it works for humans for humans you don't have one human who knows everything instead you have humans who specialize in different”

“different disciplines and so forth and the models have only a finite amount of parameters maybe you just can't like pack all that knowledge into this finite amount of parameters and you need to have specialized ais for different things and and in fact for small enough models that is true and like for really tiny models you just can't teach them all the things that they currently know and so you would need to have a specialized model for different things but what we've learned empirically is that for big enough models you can just train them on everything and then they learn everything at once and”