The paper presents a novel method called Geometric Harmonization (GH) to address the issue of representation learning disparity in self-supervised learning (SSL), particularly under long-tailed distribution scenarios. Traditional SSL methods tend to prioritize dominating classes, leading to a collapse of underrepresented classes. GH works by promoting category-level uniformity, which benefits minority classes without significantly affecting the majority. The method calculates the population statistics of the embedding space in SSL and applies an instance-wise calibration to control the space expansion of dominating classes and prevent the collapse of underrepresented ones. The researchers demonstrate the effectiveness of GH across multiple benchmark datasets, showing its high tolerance to distribution skewness.
Publication date: 26 Oct 2023
Project Page: https://github.com/MediaBrain-SJTU/Geometric-Harmonization
Paper: https://arxiv.org/pdf/2310.17622
