Momentum contrastive models such as MoCo approximate very large patch sizes by maintaining a buffer over time.

“huge patch sizes and then there is like follow-up work like the the moco uh momentum contrastive models which essentially say well we can simulate huge patch sizes by effectively taking let's say a buffer over time”