Corrupted labels and class imbalance are commonly encountered in practically
collected training data, which easily leads to over-fitting of deep neural
networks (DNNs). Existing approaches alleviate these issues by adopting a
sample re-weighting strategy, which is to re-weight sample by designing
weighting function. However, it is only applicable for training data containing
only either one type of data biases. In practice, however, biased samples with
corrupted labels and of tailed classes commonly co-exist in training data. How
to handle them simultaneously is a key but under-explored problem. In this
paper, we find that these two types of biased samples, though have similar
transient loss, have distinguishable trend and characteristics in loss curves,
which could provide valuable priors for sample weight assignment. Motivated by
this, we delve into the loss curves and propose a novel probe-and-allocate
training strategy: In the probing stage, we train the network on the whole
biased training data without intervention, and record the loss curve of each
sample as an additional attribute; In the allocating stage, we feed the
resulting attribute to a newly designed curve-perception network, named
CurveNet, to learn to identify the bias type of each sample and assign proper
weights through meta-learning adaptively. The training speed of meta learning
also blocks its application. To solve it, we propose a method named skip layer
meta optimization (SLMO) to accelerate training speed by skipping the bottom
layers. Extensive synthetic and real experiments well validate the proposed
method, which achieves state-of-the-art performance on multiple challenging
benchmarks.

该论文提出了一种名为 CurveNet 的曲线感知网络，并采用探测和分配的方法，通过元学习自适应地分配适当的样本权重，以解决深度神经网络中常见的数据偏见和样本不平衡问题，并利用 skip layer meta optimization 方法加速了训练速度，该方法在多项具有挑战性的评估指标上取得了最先进的性能。

深入研究样本损失曲线以应对嘈杂和不平衡的数据

Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data

Label noise in real-world datasets encodes wrong correlation patterns and
impairs the generalization of deep neural networks (DNNs). It is critical to
find efficient ways to detect corrupted patterns. Current methods primarily
focus on designing robust training techniques to prevent DNNs from memorizing
corrupted patterns. These approaches often require customized training
processes and may overfit corrupted patterns, leading to a performance drop in
detection. In this paper, from a more data-centric perspective, we propose a
training-free solution to detect corrupted labels. Intuitively, ``closer''
instances are more likely to share the same clean label. Based on the
neighborhood information, we propose two methods: the first one uses ``local
voting" via checking the noisy label consensuses of nearby features. The second
one is a ranking-based approach that scores each instance and filters out a
guaranteed number of instances that are likely to be corrupted. We
theoretically analyze how the quality of features affects the local voting and
provide guidelines for tuning neighborhood size. We also prove the worst-case
error bound for the ranking-based method. Experiments with both synthetic and
real-world label noise demonstrate our training-free solutions consistently and
significantly improve most of the training-based baselines. Code is available
at github.com/UCSC-REAL/SimiFeat.

提出了一种从更加数据中心的视角出发的培训自由的解决方法，以检测标签上的错误。通过邻域信息，提出了基于本地投票和基于排名的方法，从而在合成和实际噪声标签上进行实验证明了该方法的可行性和有效性。