3D convolution is powerful for video classification but often computationally
expensive, recent studies mainly focus on decomposing it on spatial-temporal
and/or channel dimensions. Unfortunately, most approaches fail to achieve a
preferable balance between convolutional efficiency and feature-interaction
sufficiency. For this reason, we propose a concise and novel Channel
Tensorization Network (CT-Net), by treating the channel dimension of input
feature as a multiplication of K sub-dimensions. On one hand, it naturally
factorizes convolution in a multiple dimension way, leading to a light
computation burden. On the other hand, it can effectively enhance feature
interaction from different channels, and progressively enlarge the 3D receptive
field of such interaction to boost classification accuracy. Furthermore, we
equip our CT-Module with a Tensor Excitation (TE) mechanism. It can learn to
exploit spatial, temporal and channel attention in a high-dimensional manner,
to improve the cooperative power of all the feature dimensions in our
CT-Module. Finally, we flexibly adapt ResNet as our CT-Net. Extensive
experiments are conducted on several challenging video benchmarks, e.g.,
Kinetics-400, Something-Something V1 and V2. Our CT-Net outperforms a number of
recent SOTA approaches, in terms of accuracy and/or efficiency. The codes and
models will be available on this https URL

本研究提出 Channel Tensorization Network (CT-Net)，通过将输入特征的通道维度视为 K 个子维度的乘积来分解卷积，从而轻量化计算量，有效增强不同通道的特征交互，并逐步扩大 3D receptive field，以提高视频分类准确率。在几个具有挑战性的视频基准中（如 Kinetics-400，Something-Something V1 和 V2）的实验中表明，相较于其他最新的方法，CT-Net 在准确性和效率方面表现更优。