Trainable layers such as convolutional building blocks are the standard
network design choices by learning parameters to capture the global context
through successive spatial operations. When designing an efficient network,
trainable layers such as the depthwise convolution is the source of efficiency
in the number of parameters and FLOPs, but there was little improvement to the
model speed in practice. This paper argues that simple built-in parameter-free
operations can be a favorable alternative to the efficient trainable layers
replacing spatial operations in a network architecture. We aim to break the
stereotype of organizing the spatial operations of building blocks into
trainable layers. Extensive experimental analyses based on layer-level studies
with fully-trained models and neural architecture searches are provided to
investigate whether parameter-free operations such as the max-pool are
functional. The studies eventually give us a simple yet effective idea for
redesigning network architectures, where the parameter-free operations are
heavily used as the main building block without sacrificing the model accuracy
as much. Experimental results on the ImageNet dataset demonstrate that the
network architectures with parameter-free operations could enjoy the advantages
of further efficiency in terms of model speed, the number of the parameters,
and FLOPs. Code and ImageNet pretrained models are available at
this https URL

该论文探讨采用内置无参操作替换空间操作的网络架构的有效性，实验结果表明，这种架构可以在不牺牲模型精度的情况下进一步提高网络速度和效率，尤其适用于 ImageNet 数据集。