Convolutional neural networks (CNNs) have achieved superior performance but
still lack clarity about the nature and properties of feature extraction. In
this paper, by analyzing the sensitivity of neural networks to frequencies and
scales, we find that neural networks not only have low- and medium-frequency
biases but also prefer different frequency bands for different classes, and the
scale of objects influences the preferred frequency bands. These observations
lead to the hypothesis that neural networks must learn the ability to extract
features at various scales and frequencies. To corroborate this hypothesis, we
propose a network architecture based on Gaussian derivatives, which extracts
features by constructing scale space and employing partial derivatives as local
feature extraction operators to separate high-frequency information. This
manually designed method of extracting features from different scales allows
our GSSDNets to achieve comparable accuracy with vanilla networks on various
datasets.

本文通过分析神经网络对频率和尺度的敏感性发现，神经网络具有不同频段类别的偏好，并且物体的尺度影响首选频段。为了证实这一假设，我们提出了基于高斯导数的网络体系结构，能够以手动方式从不同尺度上提取特征，提高了在各种数据集上的可比准确性。

特征提取的频率和尺度视角

Frequency and Scale Perspectives of Feature Extraction

To what extent are two images picturing the same 3D surfaces? Even when this
is a known scene, the answer typically requires an expensive search across
scale space, with matching and geometric verification of large sets of local
features. This expense is further multiplied when a query image is evaluated
against a gallery, e.g. in visual relocalization. While we don't obviate the
need for geometric verification, we propose an interpretable image-embedding
that cuts the search in scale space to essentially a lookup.
Our approach measures the asymmetric relation between two images. The model
then learns a scene-specific measure of similarity, from training examples with
known 3D visible-surface overlaps. The result is that we can quickly identify,
for example, which test image is a close-up version of another, and by what
scale factor. Subsequently, local features need only be detected at that scale.
We validate our scene-specific model by showing how this embedding yields
competitive image-matching results, while being simpler, faster, and also
interpretable by humans.

本文介绍了一种可解释的图像嵌入方法，通过对已知具有 3D 可视表面重叠的训练示例进行测量，从而学习场景特定的相似度，并通过简单且快速的方式减少了在不同比例下的查询图像匹配和几何验证中的搜索成本。

可解释的非度量盒嵌入预测图像之间的视觉重叠

Predicting Visual Overlap of Images Through Interpretable Non-Metric Box  Embeddings

We propose a multigrid extension of convolutional neural networks (CNNs).
Rather than manipulating representations living on a single spatial grid, our
network layers operate across scale space, on a pyramid of grids. They consume
multigrid inputs and produce multigrid outputs; convolutional filters
themselves have both within-scale and cross-scale extent. This aspect is
distinct from simple multiscale designs, which only process the input at
different scales. Viewed in terms of information flow, a multigrid network
passes messages across a spatial pyramid. As a consequence, receptive field
size grows exponentially with depth, facilitating rapid integration of context.
Most critically, multigrid structure enables networks to learn internal
attention and dynamic routing mechanisms, and use them to accomplish tasks on
which modern CNNs fail.
Experiments demonstrate wide-ranging performance advantages of multigrid. On
CIFAR and ImageNet classification tasks, flipping from a single grid to
multigrid within the standard CNN paradigm improves accuracy, while being
compute and parameter efficient. Multigrid is independent of other
architectural choices; we show synergy in combination with residual
connections. Multigrid yields dramatic improvement on a synthetic semantic
segmentation dataset. Most strikingly, relatively shallow multigrid networks
can learn to directly perform spatial transformation tasks, where, in contrast,
current CNNs fail. Together, our results suggest that continuous evolution of
features on a multigrid pyramid is a more powerful alternative to existing CNN
designs on a flat grid.

本研究提出了一种基于多重网格结构的卷积神经网络，能够在不同尺度空间上操作输入和输出，实现任务的动态路由和内部注意力机制，以及通过逐级概括将上下文信息融合到网络中。在分类和语义分割等多项任务中得到了相对较好的表现表明，相对于传统的基于单一网格的卷积神经网络结构，基于多重网格的设计方案更加高效且灵活。