Transformer models have made tremendous progress in various fields in recent
years. In the field of computer vision, vision transformers (ViTs) also become
strong alternatives to convolutional neural networks (ConvNets), yet they have
not been able to replace ConvNets since both have their own merits. For
instance, ViTs are good at extracting global features with attention mechanisms
while ConvNets are more efficient in modeling local relationships due to their
strong inductive bias. A natural idea that arises is to combine the strengths
of both ConvNets and ViTs to design new structures. In this paper, we propose a
new basic neural network operator named position-aware circular convolution
(ParC) and its accelerated version Fast-ParC. The ParC operator can capture
global features by using a global kernel and circular convolution while keeping
location sensitiveness by employing position embeddings. Our Fast-ParC further
reduces the O(n2) time complexity of ParC to O(n log n) using Fast Fourier
Transform. This acceleration makes it possible to use global convolution in the
early stages of models with large feature maps, yet still maintains the overall
computational cost comparable with using 3x3 or 7x7 kernels. The proposed
operation can be used in a plug-and-play manner to 1) convert ViTs to
pure-ConvNet architecture to enjoy wider hardware support and achieve higher
inference speed; 2) replacing traditional convolutions in the deep stage of
ConvNets to improve accuracy by enlarging the effective receptive field.
Experiment results show that our ParC op can effectively enlarge the receptive
field of traditional ConvNets, and adopting the proposed op benefits both ViTs
and ConvNet models on all three popular vision tasks, image classification,
object

本研究在计算机视觉领域中，提出了一种新的神经网络算子，名为 “位置感知循环卷积（ParC）”，通过使用全局内核和循环卷积捕捉全局特征，同时采用位置嵌入来保持位置敏感性，进一步提出了其加速版本 Fast-ParC，通过 FFT 算法将复杂度 O (n2) 的 ParC 算子的时间复杂度降至 O (nlogn)，这对于生成大尺度特征映射的模型在前期阶段使用全局卷积非常有用，但是仍然保持与使用 3x3 或 7x7 卷积核的总体计算成本相当。该算子可在 ViTs 和 ConvNet 模型中使用，并在各种视觉任务中取得了有效结果，同时扩大了传统 ConvNets 的有效感受野，提高了精度。