Power Normalizations (PN) are useful non-linear operators which tackle
feature imbalances in classification problems. We study PNs in the deep
learning setup via a novel PN layer pooling feature maps. Our layer combines
the feature vectors and their respective spatial locations in the feature maps
produced by the last convolutional layer of CNN into a positive definite matrix
with second-order statistics to which PN operators are applied, forming
so-called Second-order Pooling (SOP). As the main goal of this paper is to
study Power Normalizations, we investigate the role and meaning of MaxExp and
Gamma, two popular PN functions. To this end, we provide probabilistic
interpretations of such element-wise operators and discover surrogates with
well-behaved derivatives for end-to-end training. Furthermore, we look at the
spectral applicability of MaxExp and Gamma by studying Spectral Power
Normalizations (SPN). We show that SPN on the autocorrelation/covariance matrix
and the Heat Diffusion Process (HDP) on a graph Laplacian matrix are closely
related, thus sharing their properties. Such a finding leads us to the
culmination of our work, a fast spectral MaxExp which is a variant of HDP for
covariances/autocorrelation matrices. We evaluate our ideas on fine-grained
recognition, scene recognition, and material classification, as well as in
few-shot learning and graph classification.

本文研究深度学习应用中的 Power Normalizations 及其函数 MaxExp 与 Gamma，提供概率解释并寻找训练中合适的替代函数，同时探究 Spectral Power Normalizations 的谱适用性与热扩散过程的关系，最终提出了基于谱的 MaxExp，并在图像和图分类等领域进行了实验验证。

精细图像、少样本图像和图形分类中的功率归一化

Power Normalizations in Fine-grained Image, Few-shot Image and Graph  Classification

Second- and higher-order statistics of data points have played an important
role in advancing the state of the art on several computer vision problems such
as the fine-grained image and scene recognition. However, these statistics need
to be passed via an appropriate pooling scheme to obtain the best performance.
Power Normalizations are non-linear activation units which enjoy
probability-inspired derivations and can be applied in CNNs. In this paper, we
propose a similarity learning network leveraging second-order information and
Power Normalizations. To this end, we propose several formulations capturing
second-order statistics and derive a sigmoid-like Power Normalizing function to
demonstrate its interpretability. Our model is trained end-to-end to learn the
similarity between the support set and query images for the problem of one- and
few-shot learning. The evaluations on Omniglot, miniImagenet and Open MIC
datasets demonstrate that this network obtains state-of-the-art results on
several few-shot learning protocols.

本研究提出了一种利用二阶信息和功率归一化的相似性学习网络，用于解决一种和少量样本学习问题，并在 Omniglot，miniImagenet 和 Open MIC 数据集上获得了最优结果。