We present SplitMixer, a simple and lightweight isotropic MLP-like
architecture, for visual recognition. It contains two types of interleaving
convolutional operations to mix information across spatial locations (spatial
mixing) and channels (channel mixing). The first one includes sequentially
applying two depthwise 1D kernels, instead of a 2D kernel, to mix spatial
information. The second one is splitting the channels into overlapping or
non-overlapping segments, with or without shared parameters, and applying our
proposed channel mixing approaches or 3D convolution to mix channel
information. Depending on design choices, a number of SplitMixer variants can
be constructed to balance accuracy, the number of parameters, and speed. We
show, both theoretically and experimentally, that SplitMixer performs on par
with the state-of-the-art MLP-like models while having a significantly lower
number of parameters and FLOPS. For example, without strong data augmentation
and optimization, SplitMixer achieves around 94% accuracy on CIFAR-10 with only
0.28M parameters, while ConvMixer achieves the same accuracy with about 0.6M
parameters. The well-known MLP-Mixer achieves 85.45% with 17.1M parameters. On
CIFAR-100 dataset, SplitMixer achieves around 73% accuracy, on par with
ConvMixer, but with about 52% fewer parameters and FLOPS. We hope that our
results spark further research towards finding more efficient vision
architectures and facilitate the development of MLP-like models. Code is
available at this https URL.

SplitMixer 是一个简单且轻量级的 MLP-like 架构，包含用于混合空间位置和通道信息的两种交替卷积操作，理论和实验证明它在准确性，参数数量和速度方面都表现出色。

SplitMixer: MLP 模型中的脂肪裁剪

SplitMixer: Fat Trimmed From MLP-like Models

In a seminal paper (Weitz, 2006), Weitz gave a deterministic fully polynomial
approximation scheme for count- ing exponentially weighted independent sets
(equivalently, approximating the partition function of the hard-core model from
statistical physics) on graphs of degree at most d, up to the critical activity
for the uniqueness of the Gibbs measure on the infinite d-regular tree. More
recently Sly (Sly, 2010) showed that this is optimal in the sense that if there
is an FPRAS for the hard-core partition function on graphs of maximum degree d
for activities larger than the critical activity on the infinite d-regular tree
then NP = RP. In this paper, we extend Weitz's approach to derive a
deterministic fully polynomial approx- imation scheme for the partition
function of the anti-ferromagnetic Ising model with arbitrary field on graphs
of maximum degree d, up to the corresponding critical point on the d-regular
tree. The main ingredient of our result is a proof that for two-state
anti-ferromagnetic spin systems on the d-regular tree, weak spatial mixing
implies strong spatial mixing. This in turn uses a message-decay argument which
extends a similar approach proposed recently for the hard-core model by
Restrepo et al (Restrepo et al, 2011) to the case of the anti-ferromagnetic
Ising model with arbitrary field. By a standard correspondence, these results
translate to arbitrary two-state anti-ferromagnetic spin systems with soft
constraints.

本文扩展 Weitz 的方法，针对最大度数为 d 的图上的反铁磁 Ising 模型的分区函数，推导出一个确定性完全多项式逼近方法，主要结果是证明了在 d - 正则树上，两态反铁磁自旋系统的弱空间混合意味着强空间混合。