Although there has been much progress in the area of facial expression
recognition (FER), most existing methods suffer when presented with images that
have been captured from viewing angles that are non-frontal and substantially
different from those used in the training process. In this paper, we propose
ViewFX, a novel view-invariant FER framework based on contrastive learning,
capable of accurately classifying facial expressions regardless of the input
viewing angles during inference. ViewFX learns view-invariant features of
expression using a proposed self-supervised contrastive loss which brings
together different views of the same subject with a particular expression in
the embedding space. We also introduce a supervised contrastive loss to push
the learnt view-invariant features of each expression away from other
expressions. Since facial expressions are often distinguished with very subtle
differences in the learned feature space, we incorporate the Barlow twins loss
to reduce the redundancy and correlations of the representations in the learned
representations. The proposed method is a substantial extension of our
previously proposed CL-MEx, which only had a self-supervised loss. We test the
proposed framework on two public multi-view facial expression recognition
datasets, KDEF and DDCF. The experiments demonstrate that our approach
outperforms previous works in the area and sets a new state-of-the-art for both
datasets while showing considerably less sensitivity to challenging angles and
the number of output labels used for training. We also perform detailed
sensitivity and ablation experiments to evaluate the impact of different
components of our model as well as its sensitivity to different parameters.

本文提出了一种基于对比学习的新颖视角不变的面部表情识别框架 ViewFX，能够准确分类面部表情，无论输入的观察角度如何。该方法利用自监督对比损失来学习表情的视角不变特征，同时引入有监督对比损失来使学得的特征与其他表情区分开。经过实验验证，该方法在公开多视角面部表情识别数据集 KDEF 和 DDCF 上表现优于以往的方法，成为了新的最佳方法，并且在面对具有挑战性的观察角度和训练样本标签数目时具有较低的敏感性。同时，还进行了详尽的敏感度和消融实验，评估了模型不同部分的影响以及对不同参数的敏感性。

面部表情识别的视觉不变表示的对比学习

Contrastive Learning of View-Invariant Representations for Facial  Expressions Recognition

Most of the existing self-supervised feature learning methods for 3D data
either learn 3D features from point cloud data or from multi-view images. By
exploring the inherent multi-modality attributes of 3D objects, in this paper,
we propose to jointly learn modal-invariant and view-invariant features from
different modalities including image, point cloud, and mesh with heterogeneous
networks for 3D data. In order to learn modal- and view-invariant features, we
propose two types of constraints: cross-modal invariance constraint and
cross-view invariant constraint. Cross-modal invariance constraint forces the
network to maximum the agreement of features from different modalities for same
objects, while the cross-view invariance constraint forces the network to
maximum agreement of features from different views of images for same objects.
The quality of learned features has been tested on different downstream tasks
with three modalities of data including point cloud, multi-view images, and
mesh. Furthermore, the invariance cross different modalities and views are
evaluated with the cross-modal retrieval task. Extensive evaluation results
demonstrate that the learned features are robust and have strong
generalizability across different tasks.

该论文提出了基于异构网络的多模式和多视角无关特征学习方法，通过两种约束条件实现特征信息的跨模式和跨视角一致性，并在三种数据模态下进行了验证。实验结果表明该方法能够提取出鲁棒性较强的高质量特征。

自监督的模态与视角不变特征学习

Self-supervised Modal and View Invariant Feature Learning

Comparing two images in a view-invariant way has been a challenging problem
in computer vision for a long time, as visual features are not stable under
large view point changes. In this paper, given a single input image of an
object, we synthesize new features for other views of the same object. To
accomplish this, we introduce an aligned set of 3D models in the same class as
the input object image. Each 3D model is represented by a set of views, and we
study the correlation of image patches between different views, seeking what we
call surrogates --- patches in one view whose feature content predicts well the
features of a patch in another view. In particular, for each patch in the novel
desired view, we seek surrogates from the observed view of the given image. For
a given surrogate, we predict that surrogate using linear combination of the
corresponding patches of the 3D model views, learn the coefficients, and then
transfer these coefficients on a per patch basis to synthesize the features of
the patch in the novel view. In this way we can create feature sets for all
views of the latent object, providing us a multi-view representation of the
object. View-invariant object comparisons are achieved simply by computing the
$L^2$ distances between the features of corresponding views. We provide
theoretical and empirical analysis of the feature synthesis process, and
evaluate the proposed view-agnostic distance (VAD) in fine-grained image
retrieval (100 object classes) and classification tasks. Experimental results
show that our synthesized features do enable view-independent comparison
between images and perform significantly better than traditional image features
in this respect.

该研究提出了一种使用 3D 模型和特征合成技术来实现物体多视角特征提取和无关视角比较的方法，并在细粒度图像检索和分类任务中表现出良好的性能。