Recent findings reveal that over-parameterized deep neural networks, trained beyond zero training-error, exhibit a distinctive structural pattern at the final layer, termed as Neural-collapse (NC). These results indicate that the final hidden-layer outputs in such networks display minimal within-class variations over the training set. While existing research extensively investigates this phenomenon under cross-entropy loss, there are fewer studies focusing on its contrastive counterpart, supervised contrastive (SC) loss. Through the lens of NC, this paper employs an analytical approach to study the solutions derived from optimizing the SC loss. We adopt the unconstrained features model (UFM) as a representative proxy for unveiling NC-related phenomena in sufficiently over-parameterized deep networks. We show that, despite the non-convexity of SC loss minimization, all local minima are global minima. Furthermore, the minimizer is unique (up to a rotation). We prove our results by formalizing a tight convex relaxation of the UFM. Finally, through this convex formulation, we delve deeper into characterizing the properties of global solutions under label-imbalanced training data.

透过神经坍塌（Neural-collapse）的模式来研究优化的对比损失（supervised contrastive loss）在超参数化的深度神经网络中产生的解决方案，该论文通过采用无约束特征模型（UFM）的分析方法，证明了所有局部最小值都是全局最小值，而最小化器是唯一的（除了旋转）。最后，通过这个凸形式，深入研究在标签不平衡的训练数据下全局解的特性。

监督对比表示学习：无约束特征的景观分析