Temporal modeling plays a crucial role in understanding video content. To
tackle this problem, previous studies built complicated temporal relations
through time sequence thanks to the development of computationally powerful
devices. In this work, we explore the potential of four simple arithmetic
operations for temporal modeling. Specifically, we first capture auxiliary
temporal cues by computing addition, subtraction, multiplication, and division
between pairs of extracted frame features. Then, we extract corresponding
features from these cues to benefit the original temporal-irrespective domain.
We term such a simple pipeline as an Arithmetic Temporal Module (ATM), which
operates on the stem of a visual backbone with a plug-andplay style. We conduct
comprehensive ablation studies on the instantiation of ATMs and demonstrate
that this module provides powerful temporal modeling capability at a low
computational cost. Moreover, the ATM is compatible with both CNNs- and
ViTs-based architectures. Our results show that ATM achieves superior
performance over several popular video benchmarks. Specifically, on
Something-Something V1, V2 and Kinetics-400, we reach top-1 accuracy of 65.6%,
74.6%, and 89.4% respectively. The code is available at
this https URL

通过计算加法、减法、乘法和除法等四种算术运算，提取帧特征产生辅助时间线索，利用这些线索从原时间无关域中提取对应特征，从而实现简化的时间建模。我们的研究表明，这种算术时间模块 (ATM) 在低计算成本下提供了强大的时间建模能力，并且与基于 CNNs 和 ViTs 的架构兼容，在几个常见视频基准测试上取得了优越的性能。