Recognition problems in long-tailed data, where the sample size per class is
heavily skewed, have recently gained importance because the distribution of the
sample size per class in a dataset is generally exponential unless the sample
size is intentionally adjusted. Various approaches have been devised to address
these problems. Recently, weight balancing, which combines well-known classical
regularization techniques with two-stage training, has been proposed. Despite
its simplicity, it is known for its high performance against existing methods
devised in various ways. However, there is a lack of understanding as to why
this approach is effective for long-tailed data. In this study, we analyze the
method focusing on neural collapse and cone effect at each training stage and
find that it can be decomposed into the increase in Fisher's discriminant ratio
of the feature extractor caused by weight decay and cross entropy loss and
implicit logit adjustment caused by weight decay and class-balanced loss. Our
analysis shows that the training method can be further simplified by reducing
the number of training stages to one while increasing accuracy.

研究了一种针对长尾数据的方法，基于权重平衡和两阶段训练结合的经典正则化技术，通过分析神经崩溃和锥效应，发现该方法是通过权重衰减和交叉熵损失以及类平衡损失调整的隐式 logit 方法增加特征提取器的 Fisher 判别比率。研究表明，通过将训练阶段的数量减少到一个并提高准确性，可以进一步简化训练方法。

探索长尾识别问题中的权重平衡

Exploring Weight Balancing on Long-Tailed Recognition Problem

In the real open world, data tends to follow long-tailed class distributions,
motivating the well-studied long-tailed recognition (LTR) problem. Naive
training produces models that are biased toward common classes in terms of
higher accuracy. The key to addressing LTR is to balance various aspects
including data distribution, training losses, and gradients in learning. We
explore an orthogonal direction, weight balancing, motivated by the empirical
observation that the naively trained classifier has "artificially" larger
weights in norm for common classes (because there exists abundant data to train
them, unlike the rare classes). We investigate three techniques to balance
weights, L2-normalization, weight decay, and MaxNorm. We first point out that
L2-normalization "perfectly" balances per-class weights to be unit norm, but
such a hard constraint might prevent classes from learning better classifiers.
In contrast, weight decay penalizes larger weights more heavily and so learns
small balanced weights; the MaxNorm constraint encourages growing small weights
within a norm ball but caps all the weights by the radius. Our extensive study
shows that both help learn balanced weights and greatly improve the LTR
accuracy. Surprisingly, weight decay, although underexplored in LTR,
significantly improves over prior work. Therefore, we adopt a two-stage
training paradigm and propose a simple approach to LTR: (1) learning features
using the cross-entropy loss by tuning weight decay, and (2) learning
classifiers using class-balanced loss by tuning weight decay and MaxNorm. Our
approach achieves the state-of-the-art accuracy on five standard benchmarks,
serving as a future baseline for long-tailed recognition.

本文提出了一种通过权重均衡技术来解决长尾识别问题的方法，其中使用了 L2 正则化、权重衰减和 MaxNorm 约束等技术，通过两阶段训练，该方法在五种标准基准测试中取得了最先进的准确性，成为长尾识别未来的基线。