The scarcity of well-annotated medical datasets requires leveraging transfer
learning from broader datasets like ImageNet or pre-trained models like CLIP.
Model soups averages multiple fine-tuned models aiming to improve performance
on In-Domain (ID) tasks and enhance robustness against Out-of-Distribution
(OOD) datasets. However, applying these methods to the medical imaging domain
faces challenges and results in suboptimal performance. This is primarily due
to differences in error surface characteristics that stem from data
complexities such as heterogeneity, domain shift, class imbalance, and
distributional shifts between training and testing phases. To address this
issue, we propose a hierarchical merging approach that involves local and
global aggregation of models at various levels based on models' hyperparameter
configurations. Furthermore, to alleviate the need for training a large number
of models in the hyperparameter search, we introduce a computationally
efficient method using a cyclical learning rate scheduler to produce multiple
models for aggregation in the weight space. Our method demonstrates significant
improvements over the model souping approach across multiple datasets (around
6% gain in HAM10000 and CheXpert datasets) while maintaining low computational
costs for model generation and selection. Moreover, we achieve better results
on OOD datasets than model soups. The code is available at
this https URL

通过采用层次化合并方法和周期性学习率调度器，本研究在医学图像领域得到显著的性能提升，比模型组合方法在多个数据集上（如 HAM10000 和 CheXpert 数据集）提高了 6%，同时保持了低的计算成本。