Modern healthcare often utilises radiographic images alongside textual
reports for diagnostics, encouraging the use of Vision-Language Self-Supervised
Learning (VL-SSL) with large pre-trained models to learn versatile medical
vision representations. However, most existing VL-SSL frameworks are trained
end-to-end, which is computation-heavy and can lose vital prior information
embedded in pre-trained encoders. To address both issues, we introduce the
backbone-agnostic Adaptor framework, which preserves medical knowledge in
pre-trained image and text encoders by keeping them frozen, and employs a
lightweight Adaptor module for cross-modal learning. Experiments on medical
image classification and segmentation tasks across three datasets reveal that
our framework delivers competitive performance while cutting trainable
parameters by over 90% compared to current pre-training approaches. Notably,
when fine-tuned with just 1% of data, Adaptor outperforms several
Transformer-based methods trained on full datasets in medical image
segmentation.

现代医疗常常使用放射影像和文字报告进行诊断，鼓励使用大型预训练模型进行视觉 - 语言自监督学习 (VL-SSL) 以学习多功能的医学视觉表示。然而，大部分现有的 VL-SSL 框架都是端到端训练的，计算成本高，并且可能丢失预训练编码器中嵌入的重要先验信息。为了解决这两个问题，我们引入了适用于任意骨干网络的 Adaptor 框架，通过保持预训练的图像和文本编码器的冻结状态，并采用轻量级的 Adaptor 模块进行跨模态学习来保留医学知识。在三个数据集上进行的医学图像分类和分割任务实验表明，与当前的预训练方法相比，我们的框架在保持可训练参数减少 90% 以上的同时，具有竞争性的性能。值得注意的是，仅使用 1% 的数据进行微调时，Adaptor 在医学图像分割方面胜过了针对完整数据集训练的几种基于 Transformer 的方法。

冻结骨干网络：一种参数高效的对比学习方法用于稳健的医疗视觉 - 语言预训练

Freeze the backbones: A Parameter-Efficient Contrastive Approach to  Robust Medical Vision-Language Pre-training

Large numbers of radiographic images are available in knee radiology
practices which could be used for training of deep learning models for
diagnosis of knee abnormalities. However, those images do not typically contain
readily available labels due to limitations of human annotations. The purpose
of our study was to develop an automated labeling approach that improves the
image classification model to distinguish normal knee images from those with
abnormalities or prior arthroplasty. The automated labeler was trained on a
small set of labeled data to automatically label a much larger set of unlabeled
data, further improving the image classification performance for knee
radiographic diagnosis. We developed our approach using 7,382 patients and
validated it on a separate set of 637 patients. The final image classification
model, trained using both manually labeled and pseudo-labeled data, had the
higher weighted average AUC (WAUC: 0.903) value and higher AUC-ROC values among
all classes (normal AUC-ROC: 0.894; abnormal AUC-ROC: 0.896, arthroplasty
AUC-ROC: 0.990) compared to the baseline model (WAUC=0.857; normal AUC-ROC:
0.842; abnormal AUC-ROC: 0.848, arthroplasty AUC-ROC: 0.987), trained using
only manually labeled data. DeLong tests show that the improvement is
significant on normal (p-value<0.002) and abnormal (p-value<0.001) images. Our
findings demonstrated that the proposed automated labeling approach
significantly improves the performance of image classification for radiographic
knee diagnosis, allowing for facilitating patient care and curation of large
knee datasets.

使用自动标注方法显著提高了放射性诊断中图像分类的性能，并促进了患者护理和大型膝部数据集的整理。

改进膝关节 X 射线影像分类：自动图像标注方法

Improving Image Classification of Knee Radiographs: An Automated Image  Labeling Approach

Radiographic images offer an alternative method for the rapid screening and
monitoring of Coronavirus Disease 2019 (COVID-19) patients. This approach is
limited by the shortage of radiology experts who can provide a timely
interpretation of these images. Motivated by this challenge, our paper proposes
a novel learning architecture, called Detail-Oriented Capsule Networks
(DECAPS), for the automatic diagnosis of COVID-19 from Computed Tomography (CT)
scans. Our network combines the strength of Capsule Networks with several
architecture improvements meant to boost classification accuracies. First,
DECAPS uses an Inverted Dynamic Routing mechanism which increases model
stability by preventing the passage of information from non-descriptive
regions. Second, DECAPS employs a Peekaboo training procedure which uses a
two-stage patch crop and drop strategy to encourage the network to generate
activation maps for every target concept. The network then uses the activation
maps to focus on regions of interest and combines both coarse and fine-grained
representations of the data. Finally, we use a data augmentation method based
on conditional generative adversarial networks to deal with the issue of data
scarcity. Our model achieves 84.3% precision, 91.5% recall, and 96.1% area
under the ROC curve, significantly outperforming state-of-the-art methods. We
compare the performance of the DECAPS model with three experienced,
well-trained thoracic radiologists and show that the architecture significantly
outperforms them. While further studies on larger datasets are required to
confirm this finding, our results imply that architectures like DECAPS can be
used to assist radiologists in the CT scan mediated diagnosis of COVID-19.

本研究提出了一种名为 “Detail-Oriented Capsule Networks (DECAPS)” 的网络结构，通过结合胶囊网络和多种架构改进，自动诊断放射学影像下的新型冠状病毒肺炎 (COVID-19)。DECAPS 使用倒置动态路由机制、Peekaboo 训练过程和基于条件生成对抗网络的数据增强方法将粗粒度和细粒度表达形式相结合，从而实现自适应特征提取和可解释性高的关注热力图生成。实验结果表明，该模型在诊断性能方面远远优于具有丰富临床经验和训练的放射科医生，同时具有潜在的临床应用和推广价值。