The buffer-over-time approach used by momentum contrastive models adds some noise but tends to work well.

“Um this of course adds a little bit of noise, but tends to work pretty well. But fundamentally I agree with Lecun that we need to go back away from these contrasted models, because contrast contrasitivity alone is not strong enough of a statement to uh to actually learn necessarily useful models. So what Lecun all says is well, we should actually go effective, we should revisit old ideas, which is try to map things in”