In the realm of resource-constrained mobile vision tasks, the pursuit of
efficiency and performance consistently drives innovation in lightweight
Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). While ViTs
excel at capturing global context through self-attention mechanisms, their
deployment in resource-limited environments is hindered by computational
complexity and latency. Conversely, lightweight CNNs are favored for their
parameter efficiency and low latency. This study investigates the complementary
advantages of CNNs and ViTs to develop a versatile vision backbone tailored for
resource-constrained applications. We introduce RepNeXt, a novel model series
integrates multi-scale feature representations and incorporates both serial and
parallel structural reparameterization (SRP) to enhance network depth and width
without compromising inference speed. Extensive experiments demonstrate
RepNeXt's superiority over current leading lightweight CNNs and ViTs, providing
advantageous latency across various vision benchmarks. RepNeXt-M4 matches
RepViT-M1.5's 82.3\% accuracy on ImageNet within 1.5ms on an iPhone 12,
outperforms its AP$^{box}$ by 1.1 on MS-COCO, and reduces parameters by 0.7M.
Codes and models are available at this https URL

本研究展示了一种针对资源受限应用的视觉骨干模型 RepNeXt，它整合了多尺度特征表示，并通过串行和并行结构重参数化（SRP）来增强网络深度和宽度，从而在各种视觉基准上提供了低延迟和卓越性能。

RepNeXt: 一种使用结构重参数化的快速多尺度卷积神经网络

RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization

Despite the decomposition of convolutional kernels for lightweight CNNs being
well studied, existing works that rely on tensor network diagrams or
hyperdimensional abstraction lack geometry intuition. This work devises a new
perspective by linking a 3D-reshaped kernel tensor to its various slice-wise
and rank-1 decompositions, permitting a straightforward connection between
various tensor approximations and efficient CNN modules. Specifically, it is
discovered that a pointwise-depthwise-pointwise (PDP) configuration constitutes
a viable construct for lightweight CNNs. Moreover, a novel link to the latest
ShiftNet is established, inspiring a first-ever shift layer pruning that
achieves nearly 50% compression with < 1% drop in accuracy for ShiftResNet.

通过将 3D 重塑的卷积核张量与其各种分片和秩 - 1 分解相连，连接了各种张量逼近方法和高效 CNN 模块之间的直接联系，同时发现点对点深度点（PDP）配置对于轻量级 CNN 构造是可行的，并建立了与最新的 ShiftNet 的新链接，启发了首次移位层修剪，能够以小于 1% 的精度下降实现近 50% 的压缩。