#contrastive-loss

A SimCLR loss builds a $2N \times 2N$ cosine-similarity matrix sim for $N$ images, each seen in two augmented views (rows $i$ and $i + N$ are the two views of image $i$):

pos = torch.roll(torch.eye(2 * N), N, 1)      # 1 where the column is the other view
neg = torch.ones(2 * N) - torch.eye(2 * N)    # 1 everywhere except the diagonal
numerator = torch.exp(sim / T)[pos.bool()]
denominator = torch.sum(torch.exp(sim / T) * neg, dim=1)
loss = torch.mean(-torch.log(numerator / denominator))

With a batch of $N = 4$ images, how many terms are summed in each row of denominator?