The speaker describes cartridges as highly compressed KV caches designed to fit a large amount of information into a limited space.

“the economics of this will pressure not necessarily weight updates to one big like common shared model but like kind of like modules that get subbed in so a very obvious example this is a law but it might be something else like you know there's been a lot of work to try and fit like an arbitrary context length into a fixed size like this is all the linear attention stuff and all that sort of stuff and like cartridges which are essentially kv caches trained to be very very compressed kv caches to fit in a lot of information”