We present an approach that combines automatic features learned by
convolutional neural networks (CNN) and handcrafted features computed by the
bag-of-visual-words (BOVW) model in order to achieve state-of-the-art results
in facial expression recognition. To obtain automatic features, we experiment
with multiple CNN architectures, pre-trained models and training procedures,
e.g. Dense-Sparse-Dense. After fusing the two types of features, we employ a
local learning framework to predict the class label for each test image. The
local learning framework is based on three steps. First, a k-nearest neighbors
model is applied in order to select the nearest training samples for an input
test image. Second, a one-versus-all Support Vector Machines (SVM) classifier
is trained on the selected training samples. Finally, the SVM classifier is
used to predict the class label only for the test image it was trained for.
Although we have used local learning in combination with handcrafted features
in our previous work, to the best of our knowledge, local learning has never
been employed in combination with deep features. The experiments on the 2013
Facial Expression Recognition (FER) Challenge data set, the FER+ data set and
the AffectNet data set demonstrate that our approach achieves state-of-the-art
results. With a top accuracy of 75.42% on FER 2013, 87.76% on the FER+, 59.58%
on AffectNet 8-way classification and 63.31% on AffectNet 7-way classification,
we surpass the state-of-the-art methods by more than 1% on all data sets.

通过使用卷积神经网络自动学习的特征和由视觉单词的袋子模型计算的手工特征相结合的方法进行面部表情识别，然后使用三步的局部学习框架进行类别标签预测，得出了目前最佳结果。