Classifying facial expressions into different categories requires capturing
regional distortions of facial landmarks. We believe that second-order
statistics such as covariance is better able to capture such distortions in
regional facial fea- tures. In this work, we explore the benefits of using a
man- ifold network structure for covariance pooling to improve facial
expression recognition. In particular, we first employ such kind of manifold
networks in conjunction with tradi- tional convolutional networks for spatial
pooling within in- dividual image feature maps in an end-to-end deep learning
manner. By doing so, we are able to achieve a recognition accuracy of 58.14% on
the validation set of Static Facial Expressions in the Wild (SFEW 2.0) and
87.0% on the vali- dation set of Real-World Affective Faces (RAF) Database.
Both of these results are the best results we are aware of. Besides, we
leverage covariance pooling to capture the tem- poral evolution of per-frame
features for video-based facial expression recognition. Our reported results
demonstrate the advantage of pooling image-set features temporally by stacking
the designed manifold network of covariance pool-ing on top of convolutional
network layers.

本研究探讨使用流形网络结构进行协方差池化对于人脸表情识别的改进，并且使用该方法在 Static Facial Expressions in the Wild（SFEW 2.0）和 Real-World Affective Faces（RAF）数据库上的准确率分别达到了 58.14% 和 87.0% 的最佳结果，同时这种池化方法也可用于视频人脸表情识别中。

面部表情识别的协方差池化

Covariance Pooling For Facial Expression Recognition

By stacking layers of convolution and nonlinearity, convolutional networks
(ConvNets) effectively learn from low-level to high-level features and
discriminative representations. Since the end goal of large-scale recognition
is to delineate complex boundaries of thousands of classes, adequate
exploration of feature distributions is important for realizing full potentials
of ConvNets. However, state-of-the-art works concentrate only on deeper or
wider architecture design, while rarely exploring feature statistics higher
than first-order. We take a step towards addressing this problem. Our method
consists in covariance pooling, instead of the most commonly used first-order
pooling, of high-level convolutional features. The main challenges involved are
robust covariance estimation given a small sample of large-dimensional features
and usage of the manifold structure of covariance matrices. To address these
challenges, we present a Matrix Power Normalized Covariance (MPN-COV) method.
We develop forward and backward propagation formulas regarding the nonlinear
matrix functions such that MPN-COV can be trained end-to-end. In addition, we
analyze both qualitatively and quantitatively its advantage over the well-known
Log-Euclidean metric. On the ImageNet 2012 validation set, by combining MPN-COV
we achieve over 4%, 3% and 2.5% gains for AlexNet, VGG-M and VGG-16,
respectively; integration of MPN-COV into 50-layer ResNet outperforms
ResNet-101 and is comparable to ResNet-152. The source code will be available
on the project page: this http URL

该论文提出了一种基于高级特征的协方差汇集方法，即 Matrix Power Normalized Covariance (MPN-COV)，并在 ImageNet 上的测试中表现出了显著的成果，特别是在 AlexNet，VGG-M 和 VGG-16 的情况下，MPN-COV 方法可以使性能提高超过 3％，并且与 ResNet-152 相媲美。