Action recognition is an important problem in multimedia understanding. This
paper addresses this problem by building an expressive compositional action
model. We model one action instance in the video with an ensemble of
spatio-temporal compositions: a number of discrete temporal anchor frames, each
of which is further decomposed to a layout of deformable parts. In this way,
our model can identify a Spatio-Temporal And-Or Graph (STAOG) to represent the
latent structure of actions e.g. triple jumping, swinging and high jumping. The
STAOG model comprises four layers: (i) a batch of leaf-nodes in bottom for
detecting various action parts within video patches; (ii) the or-nodes over
bottom, i.e. switch variables to activate their children leaf-nodes for
structural variability; (iii) the and-nodes within an anchor frame for
verifying spatial composition; and (iv) the root-node at top for aggregating
scores over temporal anchor frames. Moreover, the contextual interactions are
defined between leaf-nodes in both spatial and temporal domains. For model
training, we develop a novel weakly supervised learning algorithm which
iteratively determines the structural configuration (e.g. the production of
leaf-nodes associated with the or-nodes) along with the optimization of
multi-layer parameters. By fully exploiting spatio-temporal compositions and
interactions, our approach handles well large intra-class action variance (e.g.
different views, individual appearances, spatio-temporal structures). The
experimental results on the challenging databases demonstrate superior
performance of our approach over other competing methods.

通过构建表达力强的组合行为模型，模拟视频中动作实例的时空组合，采用弱监督学习算法，识别行为的潜在结构，最终实验结果表明该方法在动作识别方面的表现优于竞争方法。