OPSD distills only the knowledge needed to match a teacher’s real-world task results, preserving limited updates instead of fully matching the teacher distribution.

“OPSD preserves this property of supervised learning,”

“where instead of slingshotting towards a teacher distribution as supervised learning would have you do,”

“you only extract the knowledge that is necessary for you to achieve the same results of the teacher on actual real-world tasks.”