Electronic Health Record (EHR) provides abundant information through various
modalities. However, learning multi-modal EHR is currently facing two major
challenges, namely, 1) data embedding and 2) cases with missing modality. A
lack of shared embedding function across modalities can discard the temporal
relationship between different EHR modalities. On the other hand, most EHR
studies are limited to relying only on EHR Times-series, and therefore, missing
modality in EHR has not been well-explored. Therefore, in this study, we
introduce a Unified Multi-modal Set Embedding (UMSE) and Modality-Aware
Attention (MAA) with Skip Bottleneck (SB). UMSE treats all EHR modalities
without a separate imputation module or error-prone carry-forward, whereas MAA
with SB learns missing modal EHR with effective modality-aware attention. Our
model outperforms other baseline models in mortality, vasopressor need, and
intubation need prediction with the MIMIC-IV dataset.

本研究提出一种统一的多模态集合嵌入和具有跳过瓶颈的模态感知注意力（MAA），在 MIMIC-IV 数据集上预测死亡率、血管加压素需求和气管插管需求中，该模型优于其他基线模型，无需单独的补偿模块或易错的向前传递。