Multimodal emotion recognition leverages complementary information across
modalities to gain performance. However, we cannot guarantee that the data of
all modalities are always present in practice. In the studies to predict the
missing data across modalities, the inherent difference between heterogeneous
modalities, namely the modality gap, presents a challenge. To address this, we
propose to use invariant features for a missing modality imagination network
(IF-MMIN) which includes two novel mechanisms: 1) an invariant feature learning
strategy that is based on the central moment discrepancy (CMD) distance under
the full-modality scenario; 2) an invariant feature based imagination module
(IF-IM) to alleviate the modality gap during the missing modalities prediction,
thus improving the robustness of multimodal joint representation. Comprehensive
experiments on the benchmark dataset IEMOCAP demonstrate that the proposed
model outperforms all baselines and invariantly improves the overall emotion
recognition performance under uncertain missing-modality conditions. We release
the code at: this https URL.

本研究提出了使用不变特征的缺失模态想象网络解决异构模态之间的模态差异问题，改善了缺失模态预测的鲁棒性并提升了多模态情感识别表现。

利用模态不变特征进行鲁棒性多模态情感识别（缺失模态）

Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities

A High-dimensional and sparse (HiDS) matrix is frequently encountered in a
big data-related application like an e-commerce system or a social network
services system. To perform highly accurate representation learning on it is of
great significance owing to the great desire of extracting latent knowledge and
patterns from it. Latent factor analysis (LFA), which represents an HiDS matrix
by learning the low-rank embeddings based on its observed entries only, is one
of the most effective and efficient approaches to this issue. However, most
existing LFA-based models perform such embeddings on a HiDS matrix directly
without exploiting its hidden graph structures, thereby resulting in accuracy
loss. To address this issue, this paper proposes a graph-incorporated latent
factor analysis (GLFA) model. It adopts two-fold ideas: 1) a graph is
constructed for identifying the hidden high-order interaction (HOI) among nodes
described by an HiDS matrix, and 2) a recurrent LFA structure is carefully
designed with the incorporation of HOI, thereby improving the representa-tion
learning ability of a resultant model. Experimental results on three real-world
datasets demonstrate that GLFA outperforms six state-of-the-art models in
predicting the missing data of an HiDS matrix, which evidently supports its
strong representation learning ability to HiDS data.

本文提出了一种基于图的潜因子分析模型，将隐藏的高阶交互与重复的低秩嵌入相结合，从而提高了对高维稀疏矩阵的表示学习能力，并在三个真实数据集上验证了其在预测丢失数据方面的优越性。

面向高维稀疏矩阵的图嵌入潜因子分析

Graph-incorporated Latent Factor Analysis for High-dimensional and Sparse Matrices

Deep generative models often perform poorly in real-world applications due to
the heterogeneity of natural data sets. Heterogeneity arises from data
containing different types of features (categorical, ordinal, continuous, etc.)
and features of the same type having different marginal distributions. We
propose an extension of variational autoencoders (VAEs) called VAEM to handle
such heterogeneous data. VAEM is a deep generative model that is trained in a
two stage manner such that the first stage provides a more uniform
representation of the data to the second stage, thereby sidestepping the
problems caused by heterogeneous data. We provide extensions of VAEM to handle
partially observed data, and demonstrate its performance in data generation,
missing data prediction and sequential feature selection tasks. Our results
show that VAEM broadens the range of real-world applications where deep
generative models can be successfully deployed.

VAEM 是 VAE 的扩展，可以处理包含不同类型特征和相同类型特征不同边缘分布的异质数据。VAEM 不仅可以用于数据生成，还可以处理缺失数据预测和序列特征选择等任务，扩展了深度生成模型的实际应用领域。