Human facial action units (AUs) are mutually related in a hierarchical
manner, as not only they are associated with each other in both spatial and
temporal domains but also AUs located in the same/close facial regions show
stronger relationships than those of different facial regions. While none of
existing approach thoroughly model such hierarchical inter-dependencies among
AUs, this paper proposes to comprehensively model multi-scale AU-related
dynamic and hierarchical spatio-temporal relationship among AUs for their
occurrences recognition. Specifically, we first propose a novel multi-scale
temporal differencing network with an adaptive weighting block to explicitly
capture facial dynamics across frames at different spatial scales, which
specifically considers the heterogeneity of range and magnitude in different
AUs' activation. Then, a two-stage strategy is introduced to hierarchically
model the relationship among AUs based on their spatial distribution (i.e.,
local and cross-region AU relationship modelling). Experimental results
achieved on BP4D and DISFA show that our approach is the new state-of-the-art
in the field of AU occurrence recognition. Our code is publicly available at
this https URL

本文提出了一种综合模型多尺度面部动作单元相关的动态和分层时空关系，以实现面部动作单元的发生识别。实验结果表明，该方法在面部动作单元发生识别领域具有新的最先进水平。

多尺度动态和分层关系建模用于面部运动单元识别

Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial  Action Units Recognition

For a given video-based Human-Object Interaction scene, modeling the
spatio-temporal relationship between humans and objects are the important cue
to understand the contextual information presented in the video. With the
effective spatio-temporal relationship modeling, it is possible not only to
uncover contextual information in each frame but also to directly capture
inter-time dependencies. It is more critical to capture the position changes of
human and objects over the spatio-temporal dimension when their appearance
features may not show up significant changes over time. The full use of
appearance features, the spatial location and the semantic information are also
the key to improve the video-based Human-Object Interaction recognition
performance. In this paper, Spatio-Temporal Interaction Graph Parsing Networks
(STIGPN) are constructed, which encode the videos with a graph composed of
human and object nodes. These nodes are connected by two types of relations:
(i) spatial relations modeling the interactions between human and the
interacted objects within each frame. (ii) inter-time relations capturing the
long range dependencies between human and the interacted objects across frame.
With the graph, STIGPN learn spatio-temporal features directly from the whole
video-based Human-Object Interaction scenes. Multi-modal features and a
multi-stream fusion strategy are used to enhance the reasoning capability of
STIGPN. Two Human-Object Interaction video datasets, including CAD-120 and
Something-Else, are used to evaluate the proposed architectures, and the
state-of-the-art performance demonstrates the superiority of STIGPN.

该论文提出了一种基于 Spatio-Temporal Interaction Graph Parsing Networks（STIGPN）的视频人物 - 物体交互识别架构，采用多模态特征融合策略，借助空间和时间关系及语义信息构建图结构，取得了比现有方法更高的准确度。