Structural re-parameterization has drawn increasing attention in various
computer vision tasks. It aims at improving the performance of deep models
without introducing any inference-time cost. Though efficient during inference,
such models rely heavily on the complicated training-time blocks to achieve
high accuracy, leading to large extra training cost. In this paper, we present
online convolutional re-parameterization (OREPA), a two-stage pipeline, aiming
to reduce the huge training overhead by squeezing the complex training-time
block into a single convolution. To achieve this goal, we introduce a linear
scaling layer for better optimizing the online blocks. Assisted with the
reduced training cost, we also explore some more effective re-param components.
Compared with the state-of-the-art re-param models, OREPA is able to save the
training-time memory cost by about 70% and accelerate the training speed by
around 2x. Meanwhile, equipped with OREPA, the models outperform previous
methods on ImageNet by up to +0.6%.We also conduct experiments on object
detection and semantic segmentation and show consistent improvements on the
downstream tasks. Codes are available at
this https URL .

本文提出了在线卷积重参数化（OREPA）方法来减少庞大的训练时间的成本，并探讨了一些更有效的重参数化组件，相较于现有的重参数化模型能够节省约 70％的训练时间内存成本，加速训练速度约 2 倍，并在 ImageNet 上优于以前的方法高达 + 0.6％，在物体检测和语义分割上也取得了一致的改善。