Real-world data usually suffers from severe class imbalance and long-tailed
distributions, where minority classes are significantly underrepresented
compared to the majority ones. Recent research prefers to utilize multi-expert
architectures to mitigate the model uncertainty on the minority, where
collaborative learning is employed to aggregate the knowledge of experts, i.e.,
online distillation. In this paper, we observe that the knowledge transfer
between experts is imbalanced in terms of class distribution, which results in
limited performance improvement of the minority classes. To address it, we
propose a re-weighted distillation loss by comparing two classifiers'
predictions, which are supervised by online distillation and label annotations,
respectively. We also emphasize that feature-level distillation will
significantly improve model performance and increase feature robustness.
Finally, we propose an Effective Collaborative Learning (ECL) framework that
integrates a contrastive proxy task branch to further improve feature quality.
Quantitative and qualitative experiments on four standard datasets demonstrate
that ECL achieves state-of-the-art performance and the detailed ablation
studies manifest the effectiveness of each component in ECL.

该论文研究类别不平衡问题，提出基于比较两个分类器预测结果的重新加权蒸馏损失，以改善专家之间的知识转移不平衡问题，并结合对比代理任务支路实现特征质量的进一步提升，实验表明所构建模型达到了最优性能。