Dynamic convolution learns a linear mixture of n static kernels weighted with
their input-dependent attentions, demonstrating superior performance than
normal convolution. However, it increases the number of convolutional
parameters by n times, and thus is not parameter efficient. This leads to no
research progress that can allow researchers to explore the setting n>100 (an
order of magnitude larger than the typical setting n<10) for pushing forward
the performance boundary of dynamic convolution while enjoying parameter
efficiency. To fill this gap, in this paper, we propose KernelWarehouse, a more
general form of dynamic convolution, which redefines the basic concepts of
``kernels", ``assembling kernels" and ``attention function" through the lens of
exploiting convolutional parameter dependencies within the same layer and
across neighboring layers of a ConvNet. We testify the effectiveness of
KernelWarehouse on ImageNet and MS-COCO datasets using various ConvNet
architectures. Intriguingly, KernelWarehouse is also applicable to Vision
Transformers, and it can even reduce the model size of a backbone while
improving the model accuracy. For instance, KernelWarehouse (n=4) achieves
5.61%|3.90%|4.38% absolute top-1 accuracy gain on the
ResNet18|MobileNetV2|DeiT-Tiny backbone, and KernelWarehouse (n=1/4) with
65.10% model size reduction still achieves 2.29% gain on the ResNet18 backbone.
The code and models are available at this https URL

KernelWarehouse 是一种更通用的动态卷积形式，通过在同一层及相邻层之间利用卷积参数的依赖关系重新定义了 “核心”、“组装核心” 和 “注意函数” 等基本概念，以提高动态卷积的参数效率，并在 ImageNet 和 MS-COCO 数据集上使用各种 ConvNet 架构测试了其有效性。KernelWarehouse 还可应用于 Vision Transformers，在提高模型准确性的同时减小主干模型的尺寸。

KernelWarehouse: 动态卷积设计的重新思考

KernelWarehouse: Rethinking the Design of Dynamic Convolution

Since the breakthrough performance of AlexNet in 2012, convolutional neural
networks (convnets) have grown into extremely powerful vision models. Deep
learning researchers have used convnets to produce accurate results that were
unachievable a decade ago. Yet computer scientists make computational
efficiency their primary objective. Accuracy with exorbitant cost is not
acceptable; an algorithm must also minimize its computational requirements.
Confronted with the daunting computation that convnets use, deep learning
researchers also became interested in efficiency. Researchers applied
tremendous effort to find the convnet architectures that have the greatest
efficiency. However, skepticism grew among researchers and engineers alike
about the relevance of arithmetic complexity. Contrary to the prevailing view
that latency and arithmetic complexity are irreconcilable, a simple formula
relates both through computational efficiency. This insight enabled us to
co-optimize the separate factors that determine latency. We observed that the
degenerate conv2d layers that produce the best accuracy-complexity trade-off
also have low operational intensity. Therefore, kernels that implement these
layers use significant memory resources. We solved this optimization problem
with block-fusion kernels that implement all layers of a residual block,
thereby creating temporal locality, avoiding communication, and reducing
workspace size. Our ConvFirst model with block-fusion kernels ran approximately
four times as fast as the ConvNeXt baseline with PyTorch Inductor, at equal
accuracy on the ImageNet-1K classification task. Our unified approach to
convnet efficiency envisions a new era of models and kernels that achieve
greater accuracy at lower cost.

通过优化卷积神经网络架构和使用融合块核心技术，实现了计算效率和准确性的提升。

卷积神经网络的效率研究

On the Efficiency of Convolutional Neural Networks

Exploiting the great expressive power of Deep Neural Network architectures,
relies on the ability to train them. While current theoretical work provides,
mostly, results showing the hardness of this task, empirical evidence usually
differs from this line, with success stories in abundance. A strong position
among empirically successful architectures is captured by networks where
extensive weight sharing is used, either by Convolutional or Recurrent layers.
Additionally, characterizing specific aspects of different tasks, making them
"harder" or "easier", is an interesting direction explored both theoretically
and empirically. We consider a family of ConvNet architectures, and prove that
weight sharing can be crucial, from an optimization point of view. We explore
different notions of the frequency, of the target function, proving necessity
of the target function having some low frequency components. This necessity is
not sufficient - only with weight sharing can it be exploited, thus
theoretically separating architectures using it, from others which do not. Our
theoretical results are aligned with empirical experiments in an even more
general setting, suggesting viability of examination of the role played by
interleaving those aspects in broader families of tasks.

本文研究卷积神经网络体系结构中共享权重对优化的重要性，探索不同目标函数频率的不同概念，并证明其必要性。研究结果表明，使用共享权重的体系结构与其他不使用共享权重的体系结构在优化方面存在理论差异。

共享权重对于成功的优化至关重要

Weight Sharing is Crucial to Succesful Optimization

Deep convolutional networks have achieved great success for visual
recognition in still images. However, for action recognition in videos, the
advantage over traditional methods is not so evident. This paper aims to
discover the principles to design effective ConvNet architectures for action
recognition in videos and learn these models given limited training samples.
Our first contribution is temporal segment network (TSN), a novel framework for
video-based action recognition. which is based on the idea of long-range
temporal structure modeling. It combines a sparse temporal sampling strategy
and video-level supervision to enable efficient and effective learning using
the whole action video. The other contribution is our study on a series of good
practices in learning ConvNets on video data with the help of temporal segment
network. Our approach obtains the state-the-of-art performance on the datasets
of HMDB51 ( $ 69.4\% $) and UCF101 ($ 94.2\% $). We also visualize the learned
ConvNet models, which qualitatively demonstrates the effectiveness of temporal
segment network and the proposed good practices.

本文提出了一种新的视频动作识别框架 - TSN，并探究了在时间段网络的帮助下学习 ConvNet 模型的一系列良好实践策略。实验结果表明，本方法在 HMDB51（69.4%）和 UCF101（94.2%）数据集上取得了最先进的性能。我们还可视化了学习到的 ConvNet 模型，定性展示了时间段网络和所提出良好实践的有效性。