For a widely-studied data model and general loss and sample-hardening
functions we prove that the Supervised Contrastive Learning (SCL), Hard-SCL
(HSCL), and Unsupervised Contrastive Learning (UCL) risks are minimized by
representations that exhibit Neural Collapse (NC), i.e., the class means form
an Equianglular Tight Frame (ETF) and data from the same class are mapped to
the same representation. We also prove that for any representation mapping, the
HSCL and Hard-UCL (HUCL) risks are lower bounded by the corresponding SCL and
UCL risks. Although the optimality of ETF is known for SCL, albeit only for
InfoNCE loss, its optimality for HSCL and UCL under general loss and hardening
functions is novel. Moreover, our proofs are much simpler, compact, and
transparent. We empirically demonstrate, for the first time, that ADAM
optimization of HSCL and HUCL risks with random initialization and suitable
hardness levels can indeed converge to the NC geometry if we incorporate
unit-ball or unit-sphere feature normalization. Without incorporating hard
negatives or feature normalization, however, the representations learned via
ADAM suffer from dimensional collapse (DC) and fail to attain the NC geometry.

对于一种广泛研究的数据模型和一般损失和样本硬化函数，我们证明了监督对比学习（SCL）、强制对比学习（HSCL）和无监督对比学习（UCL）的风险在表现出神经塌缩（NC）的表示下被最小化，即类别平均值形成一个等角紧框架（ETF），同一类别的数据映射到相同的表示。我们还证明了对于任何表示映射，相应的 HSCL 和 Hard-UCL（HUCL）的风险都被相应的 SCL 和 UCL 的风险下界约束。虽然 ETF 对于 SCL 的最优性已知，但只在 InfoNCE 损失情况下有效，然而在一般损失和硬化函数下，对于 HSCL 和 UCL 的最优性是新颖的。此外，我们的证明更为简洁、紧凑且透明。我们通过实验证明，初次尝试使用随机初始化和适当的硬度级别下的 HSCL 和 HUCL 风险的 ADAM 优化，结合单位球或单位球面特征归一化，确实可以收敛到 NC 几何形状。然而，不加入硬负样本或特征归一化的情况下，通过 ADAM 学习的表示存在维度塌缩（DC），并不能达到 NC 几何形状。