In the long-tailed recognition field, the Decoupled Training paradigm has
demonstrated remarkable capabilities among various methods. This paradigm
decouples the training process into separate representation learning and
classifier re-training. Previous works have attempted to improve both stages
simultaneously, making it difficult to isolate the effect of classifier
re-training. Furthermore, recent empirical studies have demonstrated that
simple regularization can yield strong feature representations, emphasizing the
need to reassess existing classifier re-training methods. In this study, we
revisit classifier re-training methods based on a unified feature
representation and re-evaluate their performances. We propose a new metric
called Logits Magnitude as a superior measure of model performance, replacing
the commonly used Weight Norm. However, since it is hard to directly optimize
the new metric during training, we introduce a suitable approximate invariant
called Regularized Standard Deviation. Based on the two newly proposed metrics,
we prove that reducing the absolute value of Logits Magnitude when it is nearly
balanced can effectively decrease errors and disturbances during training,
leading to better model performance. Motivated by these findings, we develop a
simple logits retargeting approach (LORT) without the requirement of prior
knowledge of the number of samples per class. LORT divides the original one-hot
label into small true label probabilities and large negative label
probabilities distributed across each class. Our method achieves
state-of-the-art performance on various imbalanced datasets, including
CIFAR100-LT, ImageNet-LT, and iNaturalist2018.

在这项研究中，我们重新评估了基于统一特征表示的分类器重新训练方法，并提出了一个称为 Logits Magnitude 的新度量作为模型性能的更好指标。通过减少平衡时 Logits Magnitude 的绝对值，可以有效减少训练过程中的错误和干扰，从而提高模型性能。基于这些发现，我们提出了一种简单的 Logits 重新目标化方法 (LORT)，无需先验知识即可将原始的 one-hot 标签分成小的真实标签概率和分布在每个类中的大的负标签概率。我们的方法在各种不平衡的数据集上取得了最先进的性能，包括 CIFAR100-LT、ImageNet-LT 和 iNaturalist2018。

重新思考长尾识别中的分类器再训练：一种简单的逻辑回归重新定位方法

Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple  Logits Retargeting Approach

Multi-domain learning (MDL) aims to train a model with minimal average risk
across multiple overlapping but non-identical domains. To tackle the challenges
of dataset bias and domain domination, numerous MDL approaches have been
proposed from the perspectives of seeking commonalities by aligning
distributions to reduce domain gap or reserving differences by implementing
domain-specific towers, gates, and even experts. MDL models are becoming more
and more complex with sophisticated network architectures or loss functions,
introducing extra parameters and enlarging computation costs. In this paper, we
propose a frustratingly easy and hyperparameter-free multi-domain learning
method named Decoupled Training(D-Train). D-Train is a tri-phase
general-to-specific training strategy that first pre-trains on all domains to
warm up a root model, then post-trains on each domain by splitting into multi
heads, and finally fine-tunes the heads by fixing the backbone, enabling
decouple training to achieve domain independence. Despite its extraordinary
simplicity and efficiency, D-Train performs remarkably well in extensive
evaluations of various datasets from standard benchmarks to applications of
satellite imagery and recommender systems.

我们提出了一种名为 Decoupled Training (D-Train) 的多领域学习方法，它通过分为多个头部进行先预训练再后训练，最后通过固定骨干网络对头部进行微调，实现了领域的独立，尽管该方法非常简单且高效，在各种数据集的广泛评估中表现出色。

解耦训练：令人沮丧的易于多域学习的回归

Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning

Federated Learning (FL) has become a popular distributed learning paradigm
that involves multiple clients training a global model collaboratively in a
data privacy-preserving manner. However, the data samples usually follow a
long-tailed distribution in the real world, and FL on the decentralized and
long-tailed data yields a poorly-behaved global model severely biased to the
head classes with the majority of the training samples. To alleviate this
issue, decoupled training has recently been introduced to FL, considering it
has achieved promising results in centralized long-tailed learning by
re-balancing the biased classifier after the instance-balanced training.
However, the current study restricts the capacity of decoupled training in
federated long-tailed learning with a sub-optimal classifier re-trained on a
set of pseudo features, due to the unavailability of a global balanced dataset
in FL. In this work, in order to re-balance the classifier more effectively, we
integrate the local real data with the global gradient prototypes to form the
local balanced datasets, and thus re-balance the classifier during the local
training. Furthermore, we introduce an extra classifier in the training phase
to help model the global data distribution, which addresses the problem of
contradictory optimization goals caused by performing classifier re-balancing
locally. Extensive experiments show that our method consistently outperforms
the existing state-of-the-art methods in various settings.

本篇研究对去中心化的 Federated Learning 中，数据长尾分布导致全局模型严重偏向于头部类别而出现的问题进行了探究和解决，提出了一种局部平衡数据集的方法，并在训练时引入了额外的分类器，同时优化分类器的平衡和全局数据分布的建模，实验证明本研究的方法在各种情况下都表现出了优异的效果。