Action Unit (AU) detection becomes essential for facial analysis. Many
proposed approaches face challenging problems in dealing with the alignments of
different face regions, in the effective fusion of temporal information, and in
training a model for multiple AU labels. To better address these problems, we
propose a deep learning framework for AU detection with region of interest
(ROI) adaptation, integrated multi-label learning, and optimal LSTM-based
temporal fusing. First, ROI cropping nets (ROI Nets) are designed to make sure
specifically interested regions of faces are learned independently; each
sub-region has a local convolutional neural network (CNN) - an ROI Net, whose
convolutional filters will only be trained for the corresponding region.
Second, multi-label learning is employed to integrate the outputs of those
individual ROI cropping nets, which learns the inter-relationships of various
AUs and acquires global features across sub-regions for AU detection. Finally,
the optimal selection of multiple LSTM layers to form the best LSTM Net is
carried out to best fuse temporal features, in order to make the AU prediction
the most accurate. The proposed approach is evaluated on two popular AU
detection datasets, BP4D and DISFA, outperforming the state of the art
significantly, with an average improvement of around 13% on BP4D and 25% on
DISFA, respectively.

提出一种深度学习框架来解决面部分析中动作单元检测中不同面部区域的对齐问题、时间信息的有效融合问题、以及多标签模型的训练问题，该方法结合了 ROI 适应性、多标签学习和基于 LSTM 的时间融合，评估结果表明，相较于现有技术在 BP4D 和 DISFA 数据集上，该方法平均提高 13％和 25％的检测准确率，取得了显著的性能提升。