While state-of-the-art facial expression recognition (FER) classifiers
achieve a high level of accuracy, they lack interpretability, an important
aspect for end-users. To recognize basic facial expressions, experts resort to
a codebook associating a set of spatial action units to a facial expression. In
this paper, we follow the same expert footsteps, and propose a learning
strategy that allows us to explicitly incorporate spatial action units (aus)
cues into the classifier's training to build a deep interpretable model. In
particular, using this aus codebook, input image expression label, and facial
landmarks, a single action units heatmap is built to indicate the most
discriminative regions of interest in the image w.r.t the facial expression. We
leverage this valuable spatial cue to train a deep interpretable classifier for
FER. This is achieved by constraining the spatial layer features of a
classifier to be correlated with \aus map. Using a composite loss, the
classifier is trained to correctly classify an image while yielding
interpretable visual layer-wise attention correlated with aus maps, simulating
the experts' decision process. This is achieved using only the image class
expression as supervision and without any extra manual annotations. Moreover,
our method is generic. It can be applied to any CNN- or transformer-based deep
classifier without the need for architectural change or adding significant
training time. Our extensive evaluation on two public benchmarks RAFDB, and
AFFECTNET datasets shows that our proposed strategy can improve layer-wise
interpretability without degrading classification performance. In addition, we
explore a common type of interpretable classifiers that rely on
Class-Activation Mapping methods (CAMs), and we show that our training
technique improves the CAM interpretability.

通过使用空间动作单元和面部标记，结合图片的分类表情作为监督，构建具有解释性的深度分类器，从而提高层级可解释性。