Spatial optimization is often overlooked in many computer vision tasks.
Filters should be able to recognize the features of an object regardless of
where it is in the image. Similarity search is a crucial task where spatial
features decide an important output. The capacity of convolution to capture
visual patterns across various locations is limited. In contrast to
convolution, the involution kernel is dynamically created at each pixel based
on the pixel value and parameters that have been learned. This study
demonstrates that utilizing a single layer of involution feature extractor
alongside a compact convolution model significantly enhances the performance of
similarity search. Additionally, we improve predictions by using the GELU
activation function rather than the ReLU. The negligible amount of weight
parameters in involution with a compact model with better performance makes the
model very useful in real-world implementations. Our proposed model is below 1
megabyte in size. We have experimented with our proposed methodology and other
models on CIFAR-10, FashionMNIST, and MNIST datasets. Our proposed method
outperforms across all three datasets.

本研究通过使用单层的 involution 特征提取器与紧凑的卷积模型相结合，显著增强了相似性搜索的性能；此外，通过使用 GELU 激活函数而非 ReLU，我们改善了预测准确性。在实际应用中，involution 模型的重量参数较少且性能更佳，使其非常有用。在 CIFAR-10、FashionMNIST 和 MNIST 数据集上，我们对我们的方法和其他模型进行了实验，结果显示我们的方法在所有三个数据集上表现优秀。

空间优化紧凑型深度度量学习模型在相似性搜索中的应用

Spatially Optimized Compact Deep Metric Learning Model for Similarity  Search

Age estimation is a classic learning problem in computer vision. Many larger
and deeper CNNs have been proposed with promising performance, such as AlexNet,
VggNet, GoogLeNet and ResNet. However, these models are not practical for the
embedded/mobile devices. Recently, MobileNets and ShuffleNets have been
proposed to reduce the number of parameters, yielding lightweight models.
However, their representation has been weakened because of the adoption of
depth-wise separable convolution. In this work, we investigate the limits of
compact model for small-scale image and propose an extremely Compact yet
efficient Cascade Context-based Age Estimation model(C3AE). This model
possesses only 1/9 and 1/2000 parameters compared with MobileNets/ShuffleNets
and VggNet, while achieves competitive performance. In particular, we re-define
age estimation problem by two-points representation, which is implemented by a
cascade model. Moreover, to fully utilize the facial context information,
multi-branch CNN network is proposed to aggregate multi-scale context.
Experiments are carried out on three age estimation datasets. The
state-of-the-art performance on compact model has been achieved with a
relatively large margin.

本研究提出了一种极其紧凑且高效的基于层叠上下文的年龄估计模型，它比 MobileNets 和 ShuffleNets 和 VggNet 只占 1/9 和 1/2000 的参数，同时具有竞争性能；通过多分支 CNN 网络聚合多尺度上下文来充分利用面部上下文信息，实现了对年龄估计问题的重新定义。

C3AE：探索紧凑模型在年龄估计方面的极限

C3AE: Exploring the Limits of Compact Model for Age Estimation

Both accuracy and efficiency are of significant importance to the task of
semantic segmentation. Existing deep FCNs suffer from heavy computations due to
a series of high-resolution feature maps for preserving the detailed knowledge
in dense estimation. Although reducing the feature map resolution (i.e.,
applying a large overall stride) via subsampling operations (e.g., pooling and
convolution striding) can instantly increase the efficiency, it dramatically
decreases the estimation accuracy. To tackle this dilemma, we propose a
knowledge distillation method tailored for semantic segmentation to improve the
performance of the compact FCNs with large overall stride. To handle the
inconsistency between the features of the student and teacher network, we
optimize the feature similarity in a transferred latent domain formulated by
utilizing a pre-trained autoencoder. Moreover, an affinity distillation module
is proposed to capture the long-range dependency by calculating the non-local
interactions across the whole image. To validate the effectiveness of our
proposed method, extensive experiments have been conducted on three popular
benchmarks: Pascal VOC, Cityscapes and Pascal Context. Built upon a highly
competitive baseline, our proposed method can improve the performance of a
student network by 2.5\% (mIOU boosts from 70.2 to 72.7 on the cityscapes test
set) and can train a better compact model with only 8\% float operations
(FLOPS) of a model that achieves comparable performances.

针对语义分割中的计算效率和精度问题，本文提出一种知识蒸馏方法，利用预训练自编码器在转移潜空间内优化特征相似度，同时引入亲和力蒸馏模块以计算整个图像的远程依赖，将经典的深度 FCN 用于大步长下进行语义分割，实验证明，所提出的方法不仅可以提高小型 FCN 的精度，还可以使用较少的 FLOPS 训练出优秀的小型模型。