In machine learning, incorporating more data is often seen as a reliable
strategy for improving model performance; this work challenges that notion by
demonstrating that the addition of external datasets in many cases can hurt the
resulting model's performance. In a large-scale empirical study across
combinations of four different open-source chest x-ray datasets and 9 different
labels, we demonstrate that in 43% of settings, a model trained on data from
two hospitals has poorer worst group accuracy over both hospitals than a model
trained on just a single hospital's data. This surprising result occurs even
though the added hospital makes the training distribution more similar to the
test distribution. We explain that this phenomenon arises from the spurious
correlation that emerges between the disease and hospital, due to
hospital-specific image artifacts. We highlight the trade-off one encounters
when training on multiple datasets, between the obvious benefit of additional
data and insidious cost of the introduced spurious correlation. In some cases,
balancing the dataset can remove the spurious correlation and improve
performance, but it is not always an effective strategy. We contextualize our
results within the literature on spurious correlations to help explain these
outcomes. Our experiments underscore the importance of exercising caution when
selecting training data for machine learning models, especially in settings
where there is a risk of spurious correlations such as with medical imaging.
The risks outlined highlight the need for careful data selection and model
evaluation in future research and practice.

在大规模实证研究中，结合四个不同的开源胸部 X 光数据集和九个不同标签的组合的情况下，通过数据来自两家医院进行训练的模型在两个医院的最差组精度方面甚至比仅使用单家医院的数据进行训练的模型更差。这种出人意料的结果是由于医院特定的图像伪相关性造成的，同时说明了训练多个数据集的权衡，即额外数据的明显好处和引入伪相关性的隐蔽成本。这些风险凸显了在未来的研究和实践中对于仔细的数据选择和模型评估的需求。

当多得更少：引入额外数据集可能会通过引入虚假关联降低性能

When More is Less: Incorporating Additional Datasets Can Hurt  Performance By Introducing Spurious Correlations

Training complex machine learning models for prediction often requires a
large amount of data that is not always readily available. Leveraging these
external datasets from related but different sources is therefore an important
task if good predictive models are to be built for deployment in settings where
data can be rare. In this paper we propose a novel approach to the problem in
which we use multiple GAN architectures to learn to translate from one dataset
to another, thereby allowing us to effectively enlarge the target dataset, and
therefore learn better predictive models than if we simply used the target
dataset. We show the utility of such an approach, demonstrating that our method
improves the prediction performance on the target domain over using just the
target dataset and also show that our framework outperforms several other
benchmarks on a collection of real-world medical datasets.

本论文提出了一种使用多个 GAN 架构进行数据集翻译的新方法，从而有效扩大目标数据集，从而在预测性能上优于仅使用目标数据集，并且在一系列真实医学数据集上优于几个其他基准。

RadialGAN: 利用生成对抗网络结合多个数据集来提高目标特定的预测模型

RadialGAN: Leveraging multiple datasets to improve target-specific  predictive models using Generative Adversarial Networks

We decompose multimodal translation into two sub-tasks: learning to translate
and learning visually grounded representations. In a multitask learning
framework, translations are learned in an attention-based encoder-decoder, and
grounded representations are learned through image representation prediction.
Our approach improves translation performance compared to the state of the art
on the Multi30K dataset. Furthermore, it is equally effective if we train the
image prediction task on the external MS COCO dataset, and we find improvements
if we train the translation model on the external News Commentary parallel
text.

本文研究了多模式翻译任务的两个子任务：学习翻译和学习视觉有关表征，并在多任务学习框架中通过注意力编码器 - 解码器和图像表征预测的方式实现。通过实验发现，这种方法在 Multi30K 数据集上比基准表现要好，即使在外部 MS COCO 数据集进行训练也同样有效，而在外部 News Commentary 平行文本训练翻译模型时进一步提高了性能。