Convolutional Neural Networks (CNNs) have been regarded as the go-to models
for visual recognition. More recently, convolution-free networks, based on
multi-head self-attention (MSA) or multi-layer perceptrons (MLPs), become more
and more popular. Nevertheless, it is not trivial when utilizing these
newly-minted networks for video recognition due to the large variations and
complexities in video data. In this paper, we present MLP-3D networks, a novel
MLP-like 3D architecture for video recognition. Specifically, the architecture
consists of MLP-3D blocks, where each block contains one MLP applied across
tokens (i.e., token-mixing MLP) and one MLP applied independently to each token
(i.e., channel MLP). By deriving the novel grouped time mixing (GTM)
operations, we equip the basic token-mixing MLP with the ability of temporal
modeling. GTM divides the input tokens into several temporal groups and
linearly maps the tokens in each group with the shared projection matrix.
Furthermore, we devise several variants of GTM with different grouping
strategies, and compose each variant in different blocks of MLP-3D network by
greedy architecture search. Without the dependence on convolutions or attention
mechanisms, our MLP-3D networks achieves 68.5\%/81.4\% top-1 accuracy on
Something-Something V2 and Kinetics-400 datasets, respectively. Despite with
fewer computations, the results are comparable to state-of-the-art widely-used
3D CNNs and video transformers. Source code is available at
this https URL.

本文使用 MLP-3D 网络结构实现了基于 token-mixing MLP 和 time mixing 的时序模型，并通过多种不同的 grouping 策略对 GTM 进行了改进，在不使用卷积或 self-attention 机制的情况下，在 Something-Something V2 和 Kinetics-400 数据集上分别实现了 68.5％/ 81.4％的 top-1 准确率，性能与 3D CNNs 和 video transformers 不相上下。

MLP-3D：一种带有分组时间混合的类 MLP 3D 架构

MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing

Recently, the proposed deep MLP models have stirred up a lot of interest in
the vision community. Historically, the availability of larger datasets
combined with increased computing capacity leads to paradigm shifts. This
review paper provides detailed discussions on whether MLP can be a new paradigm
for computer vision. We compare the intrinsic connections and differences
between convolution, self-attention mechanism, and Token-mixing MLP in detail.
Advantages and limitations of Token-mixing MLP are provided, followed by
careful analysis of recent MLP-like variants, from module design to network
architecture, and their applications. In the GPU era, the locally and globally
weighted summations are the current mainstreams, represented by the convolution
and self-attention mechanism, as well as MLP. We suggest the further
development of paradigm to be considered alongside the next-generation
computing devices.

本文综述了深度 MLP 模型在计算机视觉领域中的应用，从卷积、自注意力机制以及 Token-mixing MLP 的内在联系和区别进行了详细比较，分析了 MLP 变种的优缺点和应用，并提出了下一代计算设备和方法的发展方向。