Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular
hemodynamics marker to detect heart failure. In clinical practice, Right Heart
Catheterization is considered a gold standard for assessing cardiac
hemodynamics while non-invasive methods are often needed to screen high-risk
patients from a large population. In this paper, we propose a multimodal
learning pipeline to predict PAWP marker. We utilize complementary information
from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and
four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal
features from CMR scans using tensor-based learning. We propose a graph
attention network to select important EHR features for prediction, where we
model subjects as graph nodes and feature relationships as graph edges using
the attention mechanism. We design four feature fusion strategies: early,
intermediate, late, and hybrid fusion. With a linear classifier and linear
fusion strategies, our pipeline is interpretable. We validate our pipeline on a
large dataset of $2,641$ subjects from our ASPIRE registry. The comparative
study against state-of-the-art methods confirms the superiority of our
pipeline. The decision curve analysis further validates that our pipeline can
be applied to screen a large population. The code is available at
this https URL

利用多模型学习方法，从心脏磁共振成像扫描和电子健康记录中提取特征，设计特征融合策略，预测肺动脉楔压 (Pulmonary Arterial Wedge Pressure) 标记，并通过验证和决策曲线分析证实其优越性。

心血管血流动力学评估的可解释多模态学习

Interpretable Multimodal Learning for Cardiovascular Hemodynamics  Assessment

Multimodal Sentiment Analysis (MSA) endeavors to understand human sentiment
by leveraging language, visual, and acoustic modalities. Despite the remarkable
performance exhibited by previous MSA approaches, the presence of inherent
multimodal heterogeneities poses a challenge, with the contribution of
different modalities varying considerably. Past research predominantly focused
on improving representation learning techniques and feature fusion strategies.
However, many of these efforts overlooked the variation in semantic richness
among different modalities, treating each modality uniformly. This approach may
lead to underestimating the significance of strong modalities while
overemphasizing the importance of weak ones. Motivated by these insights, we
introduce a Text-oriented Cross-Attention Network (TCAN), emphasizing the
predominant role of the text modality in MSA. Specifically, for each multimodal
sample, by taking unaligned sequences of the three modalities as inputs, we
initially allocate the extracted unimodal features into a visual-text and an
acoustic-text pair. Subsequently, we implement self-attention on the text
modality and apply text-queried cross-attention to the visual and acoustic
modalities. To mitigate the influence of noise signals and redundant features,
we incorporate a gated control mechanism into the framework. Additionally, we
introduce unimodal joint learning to gain a deeper understanding of homogeneous
emotional tendencies across diverse modalities through backpropagation.
Experimental results demonstrate that TCAN consistently outperforms
state-of-the-art MSA methods on two datasets (CMU-MOSI and CMU-MOSEI).

多模态情感分析的研究中，我们引入了一个以文本为导向的交叉注意力网络（TCAN），强调文本在情感分析中的主导作用。通过对多模态样本进行自注意力和文本查询交叉注意力操作，我们减少了噪声信号和冗余特征的影响，并通过背向传播的方法实现了异质情感倾向的深入理解。实验证明，TCAN 在 CMU-MOSI 和 CMU-MOSEI 两个数据集上始终优于最先进的多模态情感分析方法。

TCAN: 面向文本的跨模态关注网络用于多模态情感分析

TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment  Analysis

This contribution presents a deep-learning method for extracting and fusing
image information acquired from different viewpoints with the aim to produce
more discriminant object features. Our approach was specifically designed to
mimic the morpho-constitutional analysis used by urologists to visually
classify kidney stones by inspecting the sections and surfaces of their
fragments. Deep feature fusion strategies improved the results of single view
extraction backbone models by more than 10\% in terms of precision of the
kidney stones classification.

该研究提出一种深度学习方法，旨在从不同视角提取和融合图像信息，以产生更有区别的目标特征，其中深度融合策略提高了单视图提取骨干模型的肾结石分类精度达 10％以上。