Deep Convolutional Networks (ConvNets) are fundamental to, besides
large-scale visual recognition, a lot of vision tasks. As the primary goal of
the ConvNets is to characterize complex boundaries of thousands of classes in a
high-dimensional space, it is critical to learn higher-order representations
for enhancing non-linear modeling capability. Recently, Global Second-order
Pooling (GSoP), plugged at the end of networks, has attracted increasing
attentions, achieving much better performance than classical, first-order
networks in a variety of vision tasks. However, how to effectively introduce
higher-order representation in earlier layers for improving non-linear
capability of ConvNets is still an open problem. In this paper, we propose a
novel network model introducing GSoP across from lower to higher layers for
exploiting holistic image information throughout a network. Given an input 3D
tensor outputted by some previous convolutional layer, we perform GSoP to
obtain a covariance matrix which, after nonlinear transformation, is used for
tensor scaling along channel dimension. Similarly, we can perform GSoP along
spatial dimension for tensor scaling as well. In this way, we can make full use
of the second-order statistics of the holistic image throughout a network. The
proposed networks are thoroughly evaluated on large-scale ImageNet-1K, and
experiments have shown that they outperformed non-trivially the counterparts
while achieving state-of-the-art results.

本文针对在深度卷积网络较早的层次有效引入二阶表示进行非线性建模的问题，提出了一种新的网络模型，并利用全局二阶池化 (GSoP) 对低到高层次进行二阶表示引入，使图像信息得到全面利用，实验结果表明，在 ImageNet-1K 数据集上，该网络模型优于同类网络，具有最先进的结果。