Although Vision Transformers (ViTs) have recently advanced computer vision
tasks significantly, an important real-world problem was overlooked: adapting
to variable input resolutions. Typically, images are resized to a fixed
resolution, such as 224x224, for efficiency during training and inference.
However, uniform input size conflicts with real-world scenarios where images
naturally vary in resolution. Modifying the preset resolution of a model may
severely degrade the performance. In this work, we propose to enhance the model
adaptability to resolution variation by optimizing the patch embedding. The
proposed method, called Multi-Scale Patch Embedding (MSPE), substitutes the
standard patch embedding with multiple variable-sized patch kernels and selects
the best parameters for different resolutions, eliminating the need to resize
the original image. Our method does not require high-cost training or
modifications to other parts, making it easy to apply to most ViT models.
Experiments in image classification, segmentation, and detection tasks
demonstrate the effectiveness of MSPE, yielding superior performance on
low-resolution inputs and performing comparably on high-resolution inputs with
existing methods.

本研究提出了一种名为 Multi-Scale Patch Embedding（MSPE）的方法，通过优化贴片嵌入，增强模型对分辨率变化的适应能力，在图像分类、分割和检测任务中通过消除对原始图像的调整，显著提高低分辨率输入的性能，并与现有方法在高分辨率输入方面表现相当。

MSPE：多尺度补丁嵌入激励视觉变换器适应任何分辨率

MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any  Resolution

In recent years, Transformer networks are beginning to replace pure
convolutional neural networks (CNNs) in the field of computer vision due to
their global receptive field and adaptability to input. However, the quadratic
computational complexity of softmax-attention limits the wide application in
image dehazing task, especially for high-resolution images. To address this
issue, we propose a new Transformer variant, which applies the Taylor expansion
to approximate the softmax-attention and achieves linear computational
complexity. A multi-scale attention refinement module is proposed as a
complement to correct the error of the Taylor expansion. Furthermore, we
introduce a multi-branch architecture with multi-scale patch embedding to the
proposed Transformer, which embeds features by overlapping deformable
convolution of different scales. The design of multi-scale patch embedding is
based on three key ideas: 1) various sizes of the receptive field; 2)
multi-level semantic information; 3) flexible shapes of the receptive field.
Our model, named Multi-branch Transformer expanded by Taylor formula
(MB-TaylorFormer), can embed coarse to fine features more flexibly at the patch
embedding stage and capture long-distance pixel interactions with limited
computational cost. Experimental results on several dehazing benchmarks show
that MB-TaylorFormer achieves state-of-the-art (SOTA) performance with a light
computational burden. The source code and pre-trained models are available at
this https URL

近年来，Transformer 网络开始取代纯卷积神经网络在计算机视觉领域的应用，但是其二次计算复杂度的 softmax-attention 限制了高分辨率图像去雾任务的广泛应用。因此，我们提出了一种新的 Transformer 变体，应用泰勒展开逼近 softmax-attention，实现了线性计算复杂度，并引入了多尺度注意力细化模块和多分支架构以及多尺度补丁嵌入方法进一步改进 Transformer 的性能。我们的模型名为 MB-TaylorFormer，在补丁嵌入阶段可以更灵活地嵌入粗糙到细粒度的特征，并以有限的计算成本捕捉远距离像素之间的相互作用。实验证明，MB-TaylorFormer 在多个去雾基准上取得了最先进的性能，并具有轻量级的计算负担。源代码和预训练模型可在此 https URL 获得。

MB-TaylorFormer：基于 Taylor 公式扩展的多分支高效 Transformer 用于图像去雾

MB-TaylorFormer: Multi-branch Efficient Transformer Expanded by Taylor  Formula for Image Dehazing

In implant prosthesis treatment, the design of surgical guide requires lots
of manual labors and is prone to subjective variations. When deep learning
based methods has started to be applied to address this problem, the space
between teeth are various and some of them might present similar texture
characteristic with the actual implant region. Both problems make a big
challenge for the implant position prediction. In this paper, we develop a
two-stream implant position regression framework (TSIPR), which consists of an
implant region detector (IRD) and a multi-scale patch embedding regression
network (MSPENet), to address this issue. For the training of IRD, we extend
the original annotation to provide additional supervisory information, which
contains much more rich characteristic and do not introduce extra labeling
costs. A multi-scale patch embedding module is designed for the MSPENet to
adaptively extract features from the images with various tooth spacing. The
global-local feature interaction block is designed to build the encoder of
MSPENet, which combines the transformer and convolution for enriched feature
representation. During inference, the RoI mask extracted from the IRD is used
to refine the prediction results of the MSPENet. Extensive experiments on a
dental implant dataset through five-fold cross-validation demonstrated that the
proposed TSIPR achieves superior performance than existing methods.

本文提出了一种基于深度学习的两流植入体位置回归框架，使用 IRP 作为植入区域探测器和 MSPENet 作为多尺度嵌入回归网络，并在牙种间各种间隙中采用了多尺度图像块来建立编码器以适应植入体位置预测问题。实验结果表明，该框架在牙种植入领域中表现优异。