In the real open world, data tends to follow long-tailed class distributions,
motivating the well-studied long-tailed recognition (LTR) problem. Naive
training produces models that are biased toward common classes in terms of
higher accuracy. The key to addressing LTR is to balance various aspects
including data distribution, training losses, and gradients in learning. We
explore an orthogonal direction, weight balancing, motivated by the empirical
observation that the naively trained classifier has "artificially" larger
weights in norm for common classes (because there exists abundant data to train
them, unlike the rare classes). We investigate three techniques to balance
weights, L2-normalization, weight decay, and MaxNorm. We first point out that
L2-normalization "perfectly" balances per-class weights to be unit norm, but
such a hard constraint might prevent classes from learning better classifiers.
In contrast, weight decay penalizes larger weights more heavily and so learns
small balanced weights; the MaxNorm constraint encourages growing small weights
within a norm ball but caps all the weights by the radius. Our extensive study
shows that both help learn balanced weights and greatly improve the LTR
accuracy. Surprisingly, weight decay, although underexplored in LTR,
significantly improves over prior work. Therefore, we adopt a two-stage
training paradigm and propose a simple approach to LTR: (1) learning features
using the cross-entropy loss by tuning weight decay, and (2) learning
classifiers using class-balanced loss by tuning weight decay and MaxNorm. Our
approach achieves the state-of-the-art accuracy on five standard benchmarks,
serving as a future baseline for long-tailed recognition.

本文提出了一种通过权重均衡技术来解决长尾识别问题的方法，其中使用了 L2 正则化、权重衰减和 MaxNorm 约束等技术，通过两阶段训练，该方法在五种标准基准测试中取得了最先进的准确性，成为长尾识别未来的基线。

通过权重平衡进行长尾识别

Long-Tailed Recognition via Weight Balancing

Deep distance metric learning (DDML), which is proposed to learn image
similarity metrics in an end-to-end manner based on the convolution neural
network, has achieved encouraging results in many computer vision
tasks.$L2$-normalization in the embedding space has been used to improve the
performance of several DDML methods. However, the commonly used Euclidean
distance is no longer an accurate metric for $L2$-normalized embedding space,
i.e., a hyper-sphere. Another challenge of current DDML methods is that their
loss functions are usually based on rigid data formats, such as the triplet
tuple. Thus, an extra process is needed to prepare data in specific formats. In
addition, their losses are obtained from a limited number of samples, which
leads to a lack of the global view of the embedding space. In this paper, we
replace the Euclidean distance with the cosine similarity to better utilize the
$L2$-normalization, which is able to attenuate the curse of dimensionality.
More specifically, a novel loss function based on the von Mises-Fisher
distribution is proposed to learn a compact hyper-spherical embedding space.
Moreover, a new efficient learning algorithm is developed to better capture the
global structure of the embedding space. Experiments for both classification
and retrieval tasks on several standard datasets show that our method achieves
state-of-the-art performance with a simpler training procedure. Furthermore, we
demonstrate that, even with a small number of convolutional layers, our model
can still obtain significantly better classification performance than the
widely used softmax loss.

本文提出了一种基于余弦相似度的 DDML 方法，以更好地利用 L2 归一化，并提出了一种基于 von Mises-Fisher 分布的新型损失函数来学习子空间结构，加上一种新的高效学习算法可以更好地捕捉嵌入空间的整体结构，并在几个标准数据集上进行分类和检索实验，证明我们的方法具有良好的性能和较简单的训练过程。