One primary focus in multimodal feature extraction is to find the
representations of individual modalities that are maximally correlated. As a
well-known measure of dependence, the Hirschfeld-Gebelein-R\'{e}nyi (HGR)
maximal correlation becomes an appealing objective because of its operational
meaning and desirable properties. However, the strict whitening constraints
formalized in the HGR maximal correlation limit its application. To address
this problem, this paper proposes Soft-HGR, a novel framework to extract
informative features from multiple data modalities. Specifically, our framework
prevents the "hard" whitening constraints, while simultaneously preserving the
same feature geometry as in the HGR maximal correlation. The objective of
Soft-HGR is straightforward, only involving two inner products, which
guarantees the efficiency and stability in optimization. We further generalize
the framework to handle more than two modalities and missing modalities. When
labels are partially available, we enhance the discriminative power of the
feature representations by making a semi-supervised adaptation. Empirical
evaluation implies that our approach learns more informative feature mappings
and is more efficient to optimize.

本文提出了 Soft-HGR 框架来从多个数据模式中提取信息特征，以解决 HGR 最大相关限制应用受限的问题。该框架不使用严格的白化约束，同时保留了与 HGR 最大相关相同的特征几何。我们进一步推广了该框架以处理多个模态和缺失模态，并进行了半监督的适应。经验证明，我们的方法学习到了更多的信息特征映射，并且更易于优化。

多模态数据中有用特征提取的高效方法

An Efficient Approach to Informative Feature Extraction from Multimodal  Data

Depression is a major mental health disorder that is rapidly affecting lives
worldwide. Depression not only impacts emotional but also physical and
psychological state of the person. Its symptoms include lack of interest in
daily activities, feeling low, anxiety, frustration, loss of weight and even
feeling of self-hatred. This report describes work done by us for Audio Visual
Emotion Challenge (AVEC) 2017 during our second year BTech summer internship.
With the increase in demand to detect depression automatically with the help of
machine learning algorithms, we present our multimodal feature extraction and
decision level fusion approach for the same. Features are extracted by
processing on the provided Distress Analysis Interview Corpus-Wizard of Oz
(DAIC-WOZ) database. Gaussian Mixture Model (GMM) clustering and Fisher vector
approach were applied on the visual data; statistical descriptors on gaze,
pose; low level audio features and head pose and text features were also
extracted. Classification is done on fused as well as independent features
using Support Vector Machine (SVM) and neural networks. The results obtained
were able to cross the provided baseline on validation data set by 17% on audio
features and 24.5% on video features.

本文介绍了一种多模态特征提取和决策层融合方法用于通过机器学习算法自动检测抑郁症，并使用支持向量机和神经网络对特征进行分类，在音频特征和视频特征上比验证数据集提供的基线分别提高了 17％和 24.5％。