Recent vision transformers, large-kernel CNNs and MLPs have attained
remarkable successes in broad vision tasks thanks to their effective
information fusion in the global scope. However, their efficient deployments,
especially on mobile devices, still suffer from noteworthy challenges due to
the heavy computational costs of self-attention mechanisms, large kernels, or
fully connected layers. In this work, we apply conventional convolution theorem
to deep learning for addressing this and reveal that adaptive frequency filters
can serve as efficient global token mixers. With this insight, we propose
Adaptive Frequency Filtering (AFF) token mixer. This neural operator transfers
a latent representation to the frequency domain via a Fourier transform and
performs semantic-adaptive frequency filtering via an elementwise
multiplication, which mathematically equals to a token mixing operation in the
original latent space with a dynamic convolution kernel as large as the spatial
resolution of this latent representation. We take AFF token mixers as primary
neural operators to build a lightweight neural network, dubbed AFFNet.
Extensive experiments demonstrate the effectiveness of our proposed AFF token
mixer and show that AFFNet achieve superior accuracy and efficiency trade-offs
compared to other lightweight network designs on broad visual tasks, including
visual recognition and dense prediction tasks.

通过将传统的卷积定理应用于深度学习，本研究揭示了自适应频率滤波器可作为高效的全局令牌混合器，进而提出了适应性频率滤波（AFF）令牌混合器。实验证明，AFFNet 在广泛的视觉任务中实现了卓越的准确性和效率平衡。

自适应频率滤波器作为高效的全局令牌混合器

Adaptive Frequency Filters As Efficient Global Token Mixers

A method for learning local affine-covariant regions is presented. We show
that maximizing geometric repeatability does not lead to local regions, a.k.a
features,that are reliably matched and this necessitates descriptor-based
learning. We explore factors that influence such learning and registration: the
loss function, descriptor type, geometric parametrization and the trade-off
between matchability and geometric accuracy and propose a novel hard
negative-constant loss function for learning of affine regions. The affine
shape estimator -- AffNet -- trained with the hard negative-constant loss
outperforms the state-of-the-art in bag-of-words image retrieval and wide
baseline stereo. The proposed training process does not require precisely
geometrically aligned patches.The source codes and trained weights are
available at this https URL

本文提出了一种用于学习局部仿射不变区域的方法，通过探索影响仿射变换学习和匹配的因素，如损失函数、描述符类型、几何参数化和可匹配性与几何精度之间的权衡，并提出了一种新的硬负常数损失函数，用于学习仿射区域。所提出的 AffNet 方法在图片检索和视差图等任务上具有先进的性能。