Self-supervised pre-training of deep learning models with contrastive learning is a widely used technique in image analysis. Current findings indicate a strong potential for contrastive pre-training on medical images. However, further research is necessary to incorporate the particular characteristics of these images. We hypothesize that the similarity of medical images hinders the success of contrastive learning in the medical imaging domain. To this end, we investigate different strategies based on deep embedding, information theory, and hashing in order to identify and reduce redundancy in medical pre-training datasets. The effect of these different reduction strategies on contrastive learning is evaluated on two pre-training datasets and several downstream classification tasks. In all of our experiments, dataset reduction leads to a considerable performance gain in downstream tasks, e.g., an AUC score improvement from 0.78 to 0.83 for the COVID CT Classification Grand Challenge, 0.97 to 0.98 for the OrganSMNIST Classification Challenge and 0.73 to 0.83 for a brain hemorrhage classification task. Furthermore, pre-training is up to nine times faster due to the dataset reduction. In conclusion, the proposed approach highlights the importance of dataset quality and provides a transferable approach to improve contrastive pre-training for classification downstream tasks on medical images.

本研究针对现有对比学习在医学影像中的应用问题，通过提出基于深度嵌入、信息论和哈希的策略，识别并减少医学预训练数据集的冗余。研究表明，数据集减少能显著提升下游分类任务的性能，同时预训练速度提高，强调了数据集质量在医学图像分类中的重要性。

Less is More: Selective Reduction of CT Data for Self-Supervised
  Pre-Training of Deep Learning Models with Contrastive Learning Improves
  Downstream Classification Performance

少即是多：通过选择性减少CT数据，实现深度学习模型的自监督预训练，基于对比学习提高下游分类性能