The speaker describes distillation as collecting some deployment data by querying a model and training a next-generation model on it, and says the same approach can also be used with one's own models.

“advantage both like obviously deploying this is basically what distillation is like they take out the models they get some of their like deployment data they get some fraction of that by like pinging the model and then they train their next generation of models on it and they can suddenly do it on their own models as well like there's no reason not”

“to whatsoever”