Batch normalization (BN) is an important technique commonly incorporated into
deep learning models to perform standardization within mini-batches. The merits
of BN in improving a model's learning efficiency can be further amplified by
applying whitening, while its drawbacks in estimating population statistics for
inference can be avoided through group normalization (GN). This paper proposes
group whitening (GW), which exploits the advantages of the whitening operation
and avoids the disadvantages of normalization within mini-batches. In addition,
we analyze the constraints imposed on features by normalization, and show how
the batch size (group number) affects the performance of batch (group)
normalized networks, from the perspective of model's representational capacity.
This analysis provides theoretical guidance for applying GW in practice.
Finally, we apply the proposed GW to ResNet and ResNeXt architectures and
conduct experiments on the ImageNet and COCO benchmarks. Results show that GW
consistently improves the performance of different architectures, with absolute
gains of $1.02\%$ $\sim$ $1.49\%$ in top-1 accuracy on ImageNet and $1.82\%$
$\sim$ $3.21\%$ in bounding box AP on COCO.

本文介绍了一种名为 Group Whitening 的新型批量归一化方法，该方法结合了白化方法和 Group Normalization 的优点，避免了普通批量归一化的缺点，并且从模型表征容量的角度，分析了批归一化的表征能力与批大小（组数）的关系，通过在 ResNet 和 ResNeXt 上的实验，验证了 Group Whitening 在不同架构中的性能优势。