Fine-grained action detection is an important task with numerous applications
in robotics and human-computer interaction. Existing methods typically utilize
a two-stage approach including extraction of local spatio-temporal features
followed by temporal modeling to capture long-term dependencies. While most
recent papers have focused on the latter (long-temporal modeling), here, we
focus on producing features capable of modeling fine-grained motion more
efficiently. We propose a novel locally-consistent deformable convolution,
which utilizes the change in receptive fields and enforces a local coherency
constraint to capture motion information effectively. Our model jointly learns
spatio-temporal features (instead of using independent spatial and temporal
streams). The temporal component is learned from the feature space instead of
pixel space, e.g. optical flow. The produced features can be flexibly used in
conjunction with other long-temporal modeling networks, e.g. ST-CNN,
DilatedTCN, and ED-TCN. Overall, our proposed approach robustly outperforms the
original long-temporal models on two fine-grained action datasets: 50 Salads
and GTEA, achieving F1 scores of 80.22% and 75.39% respectively.

本文提出了一种新颖的局部一致可变卷积，利用感受野变化并强制执行局部连贯性约束以有效捕获运动信息，其中时空组件是从特征空间而非像素空间学习的。最终结果表明，所提出的方法在两个动作细粒度数据集（Salads50 和 GTEA）上均显著优于原有的长期时间建模方法，分别取得了 80.22％和 75.39％的 F1 分数。