Convolutional neural networks have spatial representations which read
patterns in the vision tasks. Squeeze and excitation links the channel wise
representations by explicitly modeling on channel level. Multi layer
perceptrons learn global representations and in most of the models it is used
often at the end after all convolutional layers to gather all the information
learned before classification. We propose a method of inducing the global
representations within channels to have better performance of the model. We
propose SaEnet, Squeeze aggregated excitation network, for learning global
channelwise representation in between layers. The proposed module takes
advantage of passing important information after squeeze by having aggregated
excitation before regaining its shape. We also introduce a new idea of having a
multibranch linear(dense) layer in the network. This learns global
representations from the condensed information which enhances the
representational power of the network. The proposed module have undergone
extensive experiments by using Imagenet and CIFAR100 datasets and compared with
closely related architectures. The analyzes results that proposed models
outputs are comparable and in some cases better than existing state of the art
architectures.

本研究提出了一种称为 SaEnet 的方法，该方法利用多层感知机和聚合激活网络构建卷积神经网络的全局通道表示，通过在层之间诱导全局表示，并在网络中引入多分支线性层，从而增强网络的表征能力。通过使用 Imagenet 和 CIFAR100 数据集进行大量实验证明，该模型的性能与现有最先进的架构相当甚至更好。

集合压缩激励网络

Squeeze aggregated excitation network

Convolutional neural networks learns spatial features and are heavily
interlinked within kernels. The SE module have broken the traditional route of
neural networks passing the entire result to next layer. Instead SE only passes
important features to be learned with its squeeze and excitation (SE) module.
We propose variations of the SE module which improvises the process of squeeze
and excitation and enhances the performance. The proposed squeezing or exciting
the layer makes it possible for having a smooth transition of layer weights.
These proposed variations also retain the characteristics of SE module. The
experimented results are carried out on residual networks and the results are
tabulated.

本文提出了 SE 模块的变体，通过改进挤压和兴奋的过程，实现了平滑的层权重过渡和强化性能，在残差网络上进行实验的结果表明，这些变体保留了 SE 模块的特性。

Squeeze and Excitation 网络的变体

Variations of Squeeze and Excitation networks

Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art
performance for segmentation tasks in computer vision and medical imaging.
Recently, computational blocks termed squeeze and excitation (SE) have been
introduced to recalibrate F-CNN feature maps both channel- and spatial-wise,
boosting segmentation performance while only minimally increasing the model
complexity. So far, the development of SE blocks has focused on 2D
architectures. For volumetric medical images, however, 3D F-CNNs are a natural
choice. In this article, we extend existing 2D recalibration methods to 3D and
propose a generic compress-process-recalibrate pipeline for easy comparison of
such blocks. We further introduce Project & Excite (PE) modules, customized for
3D networks. In contrast to existing modules, Project \& Excite does not
perform global average pooling but compresses feature maps along different
spatial dimensions of the tensor separately to retain more spatial information
that is subsequently used in the excitation step. We evaluate the modules on
two challenging tasks, whole-brain segmentation of MRI scans and whole-body
segmentation of CT scans. We demonstrate that PE modules can be easily
integrated into 3D F-CNNs, boosting performance up to 0.3 in Dice Score and
outperforming 3D extensions of other recalibration blocks, while only
marginally increasing the model complexity. Our code is publicly available on
this https URL .

本文介绍了一个新颖的通用操作流程来在 3D 卷积神经网络中扩展现有的 2D 重新校准方法，并提出了专为 3D 网络定制的 Project & Excite（PE）模块，使其能更好地保留空间信息并实现对医学图像的分割性能提升。

使用 Project & Excite 重新校准 3D ConvNets

Recalibrating 3D ConvNets with Project & Excite

Attention mechanisms, and most prominently self-attention, are a powerful
building block for processing not only text but also images. These provide a
parameter efficient method for aggregating inputs. We focus on self-attention
in vision models, and we combine it with convolution, which as far as we know,
are the first to do. What emerges is a convolution with data dependent filters.
We call this an Affine Self Convolution. While this is applied differently at
each spatial location, we show that it is translation equivariant. We also
modify the Squeeze and Excitation variant of attention, extending both variants
of attention to the roto-translation group. We evaluate these new models on
CIFAR10 and CIFAR100 and show an improvement in the number of parameters, while
reaching comparable or higher accuracy at test time against self-trained
baselines.

使用数据相关卷积和注意力机制，提出了一种特殊的自注意力网络模型，称之为 Affine Self Convolution，并在 CIFAR10 和 CIFAR100 数据集上进行了评估，结果显示该模型在减少参数数量的同时，在测试时获得了与基线模型相当或更高的准确率。