Contrastive learning can be framed as pulling similar examples together and pushing dissimilar examples apart, with a loss that functions like cross-entropy over a batch.

“One of the so one of the first ideas was okay We can phrase this as kind of making similar things similar and dissimilar things dissimilar, right? And this gives you ultimately a contrastive style model by basically what seems to our effective it does is if you look at the loss effectively It's like a cross entropy,”

“but over the batch”