In this paper we revisit large kernel design in modern convolutional neural networks (CNNs), which is often neglected in the past few years. Inspired by recent advances of vision transformers (ViTs), we point out that using a few large kernels instead of a stack of small convolutions could be a more powerful paradigm. We therefore summarize 5 guidelines, e.g., applying re-parameterized large depth-wise convolutions, to design efficient high-performance large-kernel CNNs. Following the guidelines, we propose RepLKNet, a pure CNN architecture whose kernel size is as large as 31x31. RepLKNet greatly bridges the performance gap between CNNs and ViTs, e.g., achieving comparable or better results than Swin Transformer on ImageNet and downstream tasks, while the latency of RepLKNet is much lower. Moreover, RepLKNet also shows feasible scalability to big data and large models, obtaining 87.8% top-1 accuracy on ImageNet and 56.0%} mIoU on ADE20K. At last, our study further suggests large-kernel CNNs share several nice properties with ViTs, e.g., much larger effective receptive fields than conventional CNNs, and higher shape bias rather than texture bias. Code & models at https://github.com/megvii-research/RepLKNet.

本文研究卷积神经网络中大核设计在现代网络中的应用，提出了五个指南以设计高效的，基于大卷积核的CNN，并使用RepLKNet网络来实现可以与ViTs相媲美的结果，具有很好的方法可扩展性和性能表现优势。

卷积神经网络中大型卷积核设计的再思考：将卷积核扩展至31x31