Online continual learning is a challenging problem where models must learn
from a non-stationary data stream while avoiding catastrophic forgetting.
Inter-class imbalance during training has been identified as a major cause of
forgetting, leading to model prediction bias towards recently learned classes.
In this paper, we theoretically analyze that inter-class imbalance is entirely
attributed to imbalanced class-priors, and the function learned from
intra-class intrinsic distributions is the Bayes-optimal classifier. To that
end, we present that a simple adjustment of model logits during training can
effectively resist prior class bias and pursue the corresponding Bayes-optimum.
Our proposed method, Logit Adjusted Softmax, can mitigate the impact of
inter-class imbalance not only in class-incremental but also in realistic
general setups, with little additional computational cost. We evaluate our
approach on various benchmarks and demonstrate significant performance
improvements compared to prior arts. For example, our approach improves the
best baseline by 4.6% on CIFAR10.

我们提出了一种在线持续学习的方法，在训练过程中通过调整模型的 logits 来抵抗类别先验偏差并追求贝叶斯最优分类器，从而有效缓解了类间不平衡对模型性能的影响。

基于调整逻辑回归的 Softmax 的在线持续学习

Online Continual Learning via Logit Adjusted Softmax

Most of object detection algorithms can be categorized into two classes:
two-stage detectors and one-stage detectors. Recently, many efforts have been
devoted to one-stage detectors for the simple yet effective architecture.
Different from two-stage detectors, one-stage detectors aim to identify
foreground objects from all candidates in a single stage. This architecture is
efficient but can suffer from the imbalance issue with respect to two aspects:
the inter-class imbalance between the number of candidates from foreground and
background classes and the intra-class imbalance in the hardness of background
candidates, where only a few candidates are hard to be identified. In this
work, we propose a novel distributional ranking (DR) loss to handle the
challenge. For each image, we convert the classification problem to a ranking
problem, which considers pairs of candidates within the image, to address the
inter-class imbalance problem. Then, we push the distributions of confidence
scores for foreground and background towards the decision boundary. After that,
we optimize the rank of the expectations of derived distributions in lieu of
original pairs. Our method not only mitigates the intra-class imbalance issue
in background candidates but also improves the efficiency for the ranking
algorithm. By merely replacing the focal loss in RetinaNet with the developed
DR loss and applying ResNet-101 as the backbone, mAP of the single-scale test
on COCO can be improved from 39.1% to 41.7% without bells and whistles, which
demonstrates the effectiveness of the proposed loss function. Code is available
at https://github.com/idstcv/DR_loss.

本篇论文提出了分布式排名（DR）损失，以在单一阶段识别前景物体时处理前景和背景类别的数量不平衡和背景候选难度的不平衡。本方法不仅缓解了背景候选的内类别不平衡问题，而且提高了排名算法的效率。将 DR 损失应用于 RetinaNet 中，应用 ResNet-101 作为主干网络，单尺度测试的 mAP 即可从 39.1% 提高到 41.7%，证明了所提出的损失函数的有效性。