Deep neural networks (DNNs) have been used to create models for many complex
analysis problems like image recognition and medical diagnosis. DNNs are a
popular tool within machine learning due to their ability to model complex
patterns and distributions. However, the performance of these networks is
highly dependent on the quality of the data used to train the models. Two
characteristics of these sets, noisy labels and training set biases, are known
to frequently cause poor generalization performance as a result of overfitting
to the training set. This paper aims to solve this problem using the approach
proposed by Ren et al. (2018) using meta-training and online weight
approximation. We will first implement a toy-problem to crudely verify the
claims made by the authors of Ren et al. (2018) and then venture into using the
approach to solve a real world problem of Skin-cancer detection using an
imbalanced image dataset.

通过元训练和在线权重逼近方法，本文旨在解决使用深度神经网络时出现的噪声标签和训练集偏差对模型表现造成的问题，并通过使用充满挑战的不平衡图像数据集进行皮肤癌检测的真实问题来验证这一方法。

深度学习中学习重新加权示例的重新实现

Reimplementation of Learning to Reweight Examples for Robust Deep  Learning

Extractive question answering (QA) models tend to exploit spurious
correlations to make predictions when a training set has unintended biases.
This tendency results in models not being generalizable to examples where the
correlations do not hold. Determining the spurious correlations QA models can
exploit is crucial in building generalizable QA models in real-world
applications; moreover, a method needs to be developed that prevents these
models from learning the spurious correlations even when a training set is
biased. In this study, we discovered that the relative position of an answer,
which is defined as the relative distance from an answer span to the closest
question-context overlap word, can be exploited by QA models as superficial
cues for making predictions. Specifically, we find that when the relative
positions in a training set are biased, the performance on examples with
relative positions unseen during training is significantly degraded. To
mitigate the performance degradation for unseen relative positions, we propose
an ensemble-based debiasing method that does not require prior knowledge about
the distribution of relative positions. We demonstrate that the proposed method
mitigates the models' reliance on relative positions using the biased and full
SQuAD dataset. We hope that this study can help enhance the generalization
ability of QA models in real-world applications.

研究发现，抽取式问答模型可能会利用表面提示预测答案，而不是真正理解问题。相对位置也可能被利用作为这种表面提示的一种方式。为了构建一个具有普适性的抽取式问答模型，有必要找出那些有可能被利用的表面提示，并采取一些方法去防止模型利用表面提示。作者提出了一种基于集成的去除偏见的方法，无需事先了解相对位置的分布。 实验证明，该方法可以缓解模型对相对位置的依赖关系，从而提高其在实际应用中的泛化能力。