Convolutional neural networks have become a main tool for solving many
machine vision and machine learning problems. A major element of these networks
is the convolution operator which essentially computes the inner product
between a weight vector and the vectorized image patches extracted by sliding a
window in the image planes of the previous layer. In this paper, we propose two
classes of surrogate functions for the inner product operation inherent in the
convolution operator and so attain two generalizations of the convolution
operator. The first one is the class of positive definite kernel functions
where their application is justified by the kernel trick. The second one is the
class of similarity measures defined based on a distance function. We justify
this by tracing back to the basic idea behind the neocognitron which is the
ancestor of CNNs. Both methods are then further generalized by allowing a
monotonically increasing function to be applied subsequently. Like any
trainable parameter in a neural network, the template pattern and the
parameters of the kernel/distance function are trained with the
back-propagation algorithm. As an aside, we use the proposed framework to
justify the use of sine activation function in CNNs. Our experiments on the
MNIST dataset show that the performance of ordinary CNNs can be achieved by
generalized CNNs based on weighted L1/L2 distances, proving the applicability
of the proposed generalization of the convolutional neural networks.

本文提出两类代理函数，用于卷积算子内积运算的通用化，第一种是正定核函数，第二种是基于距离函数定义的相似性测量。通过在 MNIST 数据集上的实验表明，基于加权 L1/L2 距离的广义 CNN 可以实现普通 CNN 的性能，证明了卷积神经网络通用化的可行性。