Normalization techniques have been widely used in the field of deep learning
due to their capability of enabling higher learning rates and are less careful
in initialization. However, the effectiveness of popular normalization
technologies is typically limited to specific areas. Unlike the standard Batch
Normalization (BN) and Layer Normalization (LN), where BN computes the mean and
variance along the (N,H,W) dimensions and LN computes the mean and variance
along the (C,H,W) dimensions (N, C, H and W are the batch, channel, spatial
height and width dimension, respectively), this paper presents a novel
normalization technique called Batch Channel Normalization (BCN). To exploit
both the channel and batch dependence and adaptively and combine the advantages
of BN and LN based on specific datasets or tasks, BCN separately normalizes
inputs along the (N, H, W) and (C, H, W) axes, then combines the normalized
outputs based on adaptive parameters. As a basic block, BCN can be easily
integrated into existing models for various applications in the field of
computer vision. Empirical results show that the proposed technique can be
seamlessly applied to various versions of CNN or Vision Transformer
architecture. The code is publicly available at
this https URL

该研究提出了一种新的规范化技术称为批次通道规范化（BCN），它结合了批次归一化（BN）和层归一化（LN）的优势，并可以应用于计算机视觉领域中的各种应用。