Self-supervised learning is crucial for clinical imaging applications, given
the lack of explicit labels in healthcare. However, conventional approaches
that rely on precise vision-language alignment are not always feasible in
complex clinical imaging modalities, such as cardiac magnetic resonance (CMR).
CMR provides a comprehensive visualization of cardiac anatomy, physiology, and
microstructure, making it challenging to interpret. Additionally, CMR reports
require synthesizing information from sequences of images and different views,
resulting in potentially weak alignment between the study and diagnosis report
pair. To overcome these challenges, we propose \textbf{CMRformer}, a multimodal
learning framework to jointly learn sequences of CMR images and associated
cardiologist's reports. Moreover, one of the major obstacles to improving CMR
study is the lack of large, publicly available datasets. To bridge this gap, we
collected a large \textbf{CMR dataset}, which consists of 13,787 studies from
clinical cases. By utilizing our proposed CMRformer and our collected dataset,
we achieved remarkable performance in real-world clinical tasks, such as CMR
image retrieval and diagnosis report retrieval. Furthermore, the learned
representations are evaluated to be practically helpful for downstream
applications, such as disease classification. Our work could potentially
expedite progress in the CMR study and lead to more accurate and effective
diagnosis and treatment.

提出了 CMRformer 作为临床心磁共振图像和心脏病学家报告的联合学习框架可用于改善医疗保健中标签缺失和视觉上难以解释的复杂成像疾病领域的实效性，同时提供一个庞大的 CMR 数据集可用于治疗和诊断的进一步研究。