Deep convolutional neural networks (CNNs) have been widely used in various
medical imaging tasks. However, due to the intrinsic locality of convolution
operation, CNNs generally cannot model long-range dependencies well, which are
important for accurately identifying or mapping corresponding breast lesion
features computed from unregistered multiple mammograms. This motivates us to
leverage the architecture of Multi-view Vision Transformers to capture
long-range relationships of multiple mammograms from the same patient in one
examination. For this purpose, we employ local Transformer blocks to separately
learn patch relationships within four mammograms acquired from two-view
(CC/MLO) of two-side (right/left) breasts. The outputs from different views and
sides are concatenated and fed into global Transformer blocks, to jointly learn
patch relationships between four images representing two different views of the
left and right breasts. To evaluate the proposed model, we retrospectively
assembled a dataset involving 949 sets of mammograms, which include 470
malignant cases and 479 normal or benign cases. We trained and evaluated the
model using a five-fold cross-validation method. Without any arduous
preprocessing steps (e.g., optimal window cropping, chest wall or pectoral
muscle removal, two-view image registration, etc.), our four-image
(two-view-two-side) Transformer-based model achieves case classification
performance with an area under ROC curve (AUC = 0.818), which significantly
outperforms AUC = 0.784 achieved by the state-of-the-art multi-view CNNs (p =
0.009). It also outperforms two one-view-two-side models that achieve AUC of
0.724 (CC view) and 0.769 (MLO view), respectively. The study demonstrates the
potential of using Transformers to develop high-performing computer-aided
diagnosis schemes that combine four mammograms.

研究使用多视角视觉 Transformer 构建计算机辅助诊断方案，在不需要繁琐的预处理步骤的情况下，基于四张乳腺 X 线照片的 Transformer 模型达到了 81.8% 的诊断准确率，优于当前多视角 CNN 的诊断准确率。