Batch Whitening is a technique that accelerates and stabilizes training by
transforming input features to have a zero mean (Centering) and a unit variance
(Scaling), and by removing linear correlation between channels (Decorrelation).
In commonly used structures, which are empirically optimized with Batch
Normalization, the normalization layer appears between convolution and
activation function. Following Batch Whitening studies have employed the same
structure without further analysis; even Batch Whitening was analyzed on the
premise that the input of a linear layer is whitened. To bridge the gap, we
propose a new Convolutional Unit that is in line with the theory, and our
method generally improves the performance of Batch Whitening. Moreover, we show
the inefficacy of the original Convolutional Unit by investigating rank and
correlation of features. As our method is employable off-the-shelf whitening
modules, we use Iterative Normalization (IterNorm), the state-of-the-art
whitening module, and obtain significantly improved performance on five image
classification datasets: CIFAR-10, CIFAR-100, CUB-200-2011, Stanford Dogs, and
ImageNet. Notably, we verify that our method improves stability and performance
of whitening when using large learning rate, group size, and iteration number.

提出了一种新的卷积单元，可用于批处理美白，并且效果更好，使用迭代归一化并应用于五个图像分类数据集上，显着提高了性能和稳定性。

通过卷积单元优化提升批量白化的泛化能力

Improving Generalization of Batch Whitening by Convolutional Unit  Optimization

We introduce a novel and generic convolutional unit, DiCE unit, that is built
using dimension-wise convolutions and dimension-wise fusion. The dimension-wise
convolutions apply light-weight convolutional filtering across each dimension
of the input tensor while dimension-wise fusion efficiently combines these
dimension-wise representations; allowing the DiCE unit to efficiently encode
spatial and channel-wise information contained in the input tensor. The DiCE
unit is simple and can be seamlessly integrated with any architecture to
improve its efficiency and performance. Compared to depth-wise separable
convolutions, the DiCE unit shows significant improvements across different
architectures. When DiCE units are stacked to build the DiCENet model, we
observe significant improvements over state-of-the-art models across various
computer vision tasks including image classification, object detection, and
semantic segmentation. On the ImageNet dataset, the DiCENet delivers 2-4%
higher accuracy than state-of-the-art manually designed models (e.g.,
MobileNetv2 and ShuffleNetv2). Also, DiCENet generalizes better to tasks (e.g.,
object detection) that are often used in resource-constrained devices in
comparison to state-of-the-art separable convolution-based efficient networks,
including neural search-based methods (e.g., MobileNetv3 and MixNet. Our source
code in PyTorch is open-source and is available at
this https URL

提出了一种新型的通用卷积单元 DiCE unit，利用按维度卷积和维度融合有效地将输入张量的空间和通道信息编码，可以无缝地集成到任何架构中以提高效率和性能，在图像分类、目标检测和语义分割等各种计算机视觉任务中，DiCENet 模型比现有技术手动设计的模型具有更高的精度和更好的泛化性能。