This paper presents a normalization mechanism called Instance-Level Meta
Normalization (ILM~Norm) to address a learning-to-normalize problem. ILM~Norm
learns to predict the normalization parameters via both the feature
feed-forward and the gradient back-propagation paths. ILM~Norm provides a meta
normalization mechanism and has several good properties. It can be easily
plugged into existing instance-level normalization schemes such as Instance
Normalization, Layer Normalization, or Group Normalization. ILM~Norm normalizes
each instance individually and therefore maintains high performance even when
small mini-batch is used. The experimental results show that ILM~Norm well
adapts to different network architectures and tasks, and it consistently
improves the performance of the original models. The code is available at
url{this https URL

该研究提出一种称为 “ILM~Norm” 的规范化机制，它通过特征前馈和梯度反向传播路径学习预测规范化参数，适用于多种网络架构和任务，能够提高模型性能。

实例级元归一化

Instance-Level Meta Normalization

We address a learning-to-normalize problem by proposing Switchable
Normalization (SN), which learns to select different normalizers for different
normalization layers of a deep neural network. SN employs three distinct scopes
to compute statistics (means and variances) including a channel, a layer, and a
minibatch. SN switches between them by learning their importance weights in an
end-to-end manner. It has several good properties. First, it adapts to various
network architectures and tasks (see Fig.1). Second, it is robust to a wide
range of batch sizes, maintaining high performance even when small minibatch is
presented (e.g. 2 images/GPU). Third, SN does not have sensitive
hyper-parameter, unlike group normalization that searches the number of groups
as a hyper-parameter. Without bells and whistles, SN outperforms its
counterparts on various challenging benchmarks, such as ImageNet, COCO,
CityScapes, ADE20K, and Kinetics. Analyses of SN are also presented. We hope SN
will help ease the usage and understand the normalization techniques in deep
learning. The code of SN has been made available in
this https URL

本文提出可切换标准化（SN），通过学习为深度神经网络的不同标准化层选择不同的标准化器来解决学习标准化问题。 SN 在计算统计量（均值和方差）时使用三个不同的范围，包括通道，层和批处理，并通过学习它们的重要性权重来在它们之间切换，从而能适应各种网络架构和任务，并且不会受到宽范围批量大小的影响，具有很强的稳健性。 SN 不需要敏感的超参数搜索，而且在各种挑战性的基准测试中具有良好的性能，例如 ImageNet，COCO，CityScapes，ADE20K 和 Kinetics。作者也进行了分析，希望 SN 能够有助于使用和理解深度学习中的标准化技术。