Many convolutional neural networks (CNNs) rely on progressive downsampling of
their feature maps to increase the network's receptive field and decrease
computational cost. However, this comes at the price of losing granularity in
the feature maps, limiting the ability to correctly understand images or
recover fine detail in dense prediction tasks. To address this, common practice
is to replace the last few downsampling operations in a CNN with dilated
convolutions, allowing to retain the feature map resolution without reducing
the receptive field, albeit increasing the computational cost. This allows to
trade off predictive performance against cost, depending on the output feature
resolution. By either regularly downsampling or not downsampling the entire
feature map, existing work implicitly treats all regions of the input image and
subsequent feature maps as equally important, which generally does not hold. We
propose an adaptive downsampling scheme that generalizes the above idea by
allowing to process informative regions at a higher resolution than less
informative ones. In a variety of experiments, we demonstrate the versatility
of our adaptive downsampling strategy and empirically show that it improves the
cost-accuracy trade-off of various established CNNs.

该论文提出了一种适应性下采样方案，通过允许以更高的分辨率处理信息丰富的区域而不是信息较少的区域，从而提高了各种已建立 CNN 的成本 - 准确性权衡效果。

卷积神经网络中的内容自适应下采样

Content-Adaptive Downsampling in Convolutional Neural Networks

Real world images often have highly imbalanced content density. Some areas
are very uniform, e.g., large patches of blue sky, while other areas are
scattered with many small objects. Yet, the commonly used successive grid
downsampling strategy in convolutional deep networks treats all areas equally.
Hence, small objects are represented in very few spatial locations, leading to
worse results in tasks such as segmentation. Intuitively, retaining more pixels
representing small objects during downsampling helps to preserve important
information. To achieve this, we propose AutoFocusFormer (AFF), a
local-attention transformer image recognition backbone, which performs adaptive
downsampling by learning to retain the most important pixels for the task.
Since adaptive downsampling generates a set of pixels irregularly distributed
on the image plane, we abandon the classic grid structure. Instead, we develop
a novel point-based local attention block, facilitated by a balanced clustering
module and a learnable neighborhood merging module, which yields
representations for our point-based versions of state-of-the-art segmentation
heads. Experiments show that our AutoFocusFormer (AFF) improves significantly
over baseline models of similar sizes.

本研究提出了一种名为 AutoFocusFormer (AFF) 的局部 - 注意力 Transformer 图像识别骨干网络，通过学习保留任务中最重要的像素来实现自适应降采样，采用点基础分割方法，使得小目标在降采样过程中得到更充分地表示，并展示它在分割任务中显著优于基线模型。