Deep learning now enables automatic and robust extraction of cardiac function
descriptors from echocardiographic sequences, such as ejection fraction or
strain. These descriptors provide fine-grained information that physicians
consider, in conjunction with more global variables from the clinical record,
to assess patients' condition. Drawing on novel transformer models applied to
tabular data (e.g., variables from electronic health records), we propose a
method that considers all descriptors extracted from medical records and
echocardiograms to learn the representation of a difficult-to-characterize
cardiovascular pathology, namely hypertension. Our method first projects each
variable into its own representation space using modality-specific approaches.
These standardized representations of multimodal data are then fed to a
transformer encoder, which learns to merge them into a comprehensive
representation of the patient through a pretext task of predicting a clinical
rating. This pretext task is formulated as an ordinal classification to enforce
a pathological continuum in the representation space. We observe the major
trends along this continuum for a cohort of 239 hypertensive patients to
describe, with unprecedented gradation, the effect of hypertension on a number
of cardiac function descriptors. Our analysis shows that i) pretrained weights
from a foundation model allow to reach good performance (83% accuracy) even
with limited data (less than 200 training samples), ii) trends across the
population are reproducible between trainings, and iii) for descriptors whose
interactions with hypertension are well documented, patterns are consistent
with prior physiological knowledge.

利用深度学习和转换器模型，我们提出了一种方法，通过预测临床评分的假设任务，联合从电子病历和超声心动图提取的全部描述符，学习表征艰难识别的心血管病理（高血压），并详细描述了高血压对多个心脏功能描述符的影响。