Deep convolutional neural networks (DCNNs) are a class of artificial neural
networks, primarily for computer vision tasks such as segmentation and
classification. Many nonlinear operations, such as activation functions and
pooling strategies, are used in DCNNs to enhance their ability to process
different signals with different tasks. Conceptional convolution, a linear
filter, is the essential component of DCNNs while nonlinear convolution is
generally implemented as higher-order Volterra filters, However, for Volterra
filtering, significant memory and computational costs pose a primary limitation
for its widespread application in DCNN applications. In this study, we propose
a novel method to perform higher-order Volterra filtering with lower memory and
computation cost in forward and backward pass in DCNN training. The proposed
method demonstrates computational advantages compared with conventional
Volterra filter implementation. Furthermore, based on the proposed method, a
new attention module called Higher-order Local Attention Block (HLA) is
proposed and tested on CIFAR-100 dataset, which shows competitive improvement
for classification task. Source code is available at:
this https URL

通过提出一种新的方法来实现低内存和计算成本的高阶 Volterra 滤波，在 DCNN 训练的前向和反向传播过程中具有计算优势，并基于该方法提出了一种名为 Higher-order Local Attention Block（HLA）的新型注意力模块，并在 CIFAR-100 数据集上进行了测试，显示了在分类任务中的有竞争力的改进。