Sample re-weighting strategies provide a promising mechanism to deal with
imperfect training data in machine learning, such as noisily labeled or
class-imbalanced data. One such strategy involves formulating a bi-level
optimization problem called the meta re-weighting problem, whose goal is to
optimize performance on a small set of perfect pivotal samples, called meta
samples. Many approaches have been proposed to efficiently solve this problem.
However, all of them assume that a perfect meta sample set is already provided
while we observe that the selections of meta sample set is performance
critical. In this paper, we study how to learn to identify such a meta sample
set from a large, imperfect training set, that is subsequently cleaned and used
to optimize performance in the meta re-weighting setting. We propose a learning
framework which reduces the meta samples selection problem to a weighted
K-means clustering problem through rigorously theoretical analysis. We propose
two clustering methods within our learning framework, Representation-based
clustering method (RBC) and Gradient-based clustering method (GBC), for
balancing performance and computational efficiency. Empirical studies
demonstrate the performance advantage of our methods over various baseline
methods.

本研究提出了一种学习框架，将元样本选择问题通过严格的理论分析降低到加权 K 均值聚类问题中，提出了两种聚类方法 RBC 和 GBC 以解决 “样本权重不均匀” 问题，并在实际应用中得到了验证。

元重置中的关键样本选择学习

Learning to Select Pivotal Samples for Meta Re-weighting

Corrupted labels and class imbalance are commonly encountered in practically
collected training data, which easily leads to over-fitting of deep neural
networks (DNNs). Existing approaches alleviate these issues by adopting a
sample re-weighting strategy, which is to re-weight sample by designing
weighting function. However, it is only applicable for training data containing
only either one type of data biases. In practice, however, biased samples with
corrupted labels and of tailed classes commonly co-exist in training data. How
to handle them simultaneously is a key but under-explored problem. In this
paper, we find that these two types of biased samples, though have similar
transient loss, have distinguishable trend and characteristics in loss curves,
which could provide valuable priors for sample weight assignment. Motivated by
this, we delve into the loss curves and propose a novel probe-and-allocate
training strategy: In the probing stage, we train the network on the whole
biased training data without intervention, and record the loss curve of each
sample as an additional attribute; In the allocating stage, we feed the
resulting attribute to a newly designed curve-perception network, named
CurveNet, to learn to identify the bias type of each sample and assign proper
weights through meta-learning adaptively. The training speed of meta learning
also blocks its application. To solve it, we propose a method named skip layer
meta optimization (SLMO) to accelerate training speed by skipping the bottom
layers. Extensive synthetic and real experiments well validate the proposed
method, which achieves state-of-the-art performance on multiple challenging
benchmarks.

该论文提出了一种名为 CurveNet 的曲线感知网络，并采用探测和分配的方法，通过元学习自适应地分配适当的样本权重，以解决深度神经网络中常见的数据偏见和样本不平衡问题，并利用 skip layer meta optimization 方法加速了训练速度，该方法在多项具有挑战性的评估指标上取得了最先进的性能。

深入研究样本损失曲线以应对嘈杂和不平衡的数据

Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data

Despite success on a wide range of problems related to vision, generative
adversarial networks (GANs) often suffer from inferior performance due to
unstable training, especially for text generation. To solve this issue, we
propose a new variational GAN training framework which enjoys superior training
stability. Our approach is inspired by a connection of GANs and reinforcement
learning under a variational perspective. The connection leads to (1)
probability ratio clipping that regularizes generator training to prevent
excessively large updates, and (2) a sample re-weighting mechanism that
improves discriminator training by downplaying bad-quality fake samples.
Moreover, our variational GAN framework can provably overcome the training
issue in many GANs that an optimal discriminator cannot provide any informative
gradient to training generator. By plugging the training approach in diverse
state-of-the-art GAN architectures, we obtain significantly improved
performance over a range of tasks, including text generation, text style
transfer, and image generation.

提出了一种基于变分的生成式对抗网络训练框架，通过概率裁剪和样本重新加权等技术，提升文本生成、文本风格转移和图像生成等多个任务的表现

使用概率比剪切和样本重新加权改进 GAN 训练

Improving GAN Training with Probability Ratio Clipping and Sample  Reweighting

We propose a multi-task learning framework to learn a joint Machine Reading
Comprehension (MRC) model that can be applied to a wide range of MRC tasks in
different domains. Inspired by recent ideas of data selection in machine
translation, we develop a novel sample re-weighting scheme to assign
sample-specific weights to the loss. Empirical study shows that our approach
can be applied to many existing MRC models. Combined with contextual
representations from pre-trained language models (such as ELMo), we achieve new
state-of-the-art results on a set of MRC benchmark datasets. We release our
code at this https URL

本研究提出了一种多任务学习框架，用于学习一个联合的机器阅读理解模型，可应用于不同领域的阅读理解任务。研究采用了样本分配权重的方法，并结合预训练语言模型（如 ELMo），在一组阅读理解数据集上实现了新的最佳效果。