While deep learning-based methods have demonstrated outstanding results in
numerous domains, some important functionalities are missing. Resolution
scalability is one of them. In this work, we introduce a novel architecture,
dubbed RESSCAL3D, providing resolution-scalable 3D semantic segmentation of
point clouds. In contrast to existing works, the proposed method does not
require the whole point cloud to be available to start inference. Once a
low-resolution version of the input point cloud is available, first semantic
predictions can be generated in an extremely fast manner. This enables early
decision-making in subsequent processing steps. As additional points become
available, these are processed in parallel. To improve performance, features
from previously computed scales are employed as prior knowledge at the current
scale. Our experiments show that RESSCAL3D is 31-62% faster than the
non-scalable baseline while keeping a limited impact on performance. To the
best of our knowledge, the proposed method is the first to propose a
resolution-scalable approach for 3D semantic segmentation of point clouds based
on deep learning.

通过深度学习技术，我们提出了一种新的架构 RESCAL3D，用于对点云进行分辨率可扩展的三维语义分割，该方法在实现极快的推断速度的同时，保持了较高的性能水平。

RESSCAL3D：点云的分辨率可扩展三维语义分割

RESSCAL3D: Resolution Scalable 3D Semantic Segmentation of Point Clouds

his paper tackles a significant challenge faced by Vision Transformers
(ViTs): their constrained scalability across different image resolutions.
Typically, ViTs experience a performance decline when processing resolutions
different from those seen during training. Our work introduces two key
innovations to address this issue. Firstly, we propose a novel module for
dynamic resolution adjustment, designed with a single Transformer block,
specifically to achieve highly efficient incremental token integration.
Secondly, we introduce fuzzy positional encoding in the Vision Transformer to
provide consistent positional awareness across multiple resolutions, thereby
preventing overfitting to any single training resolution. Our resulting model,
ViTAR (Vision Transformer with Any Resolution), demonstrates impressive
adaptability, achieving 83.3\% top-1 accuracy at a 1120x1120 resolution and
80.4\% accuracy at a 4032x4032 resolution, all while reducing computational
costs. ViTAR also shows strong performance in downstream tasks such as instance
and semantic segmentation and can easily combined with self-supervised learning
techniques like Masked AutoEncoder. Our work provides a cost-effective solution
for enhancing the resolution scalability of ViTs, paving the way for more
versatile and efficient high-resolution image processing.

通过动态分辨率调整和模糊位置编码，ViTAR（任意分辨率的视觉 Transformer）实现了对不同分辨率图像的高效处理和一致的位置感知，提升了 ViT（视觉 Transformer）的分辨率可扩展性和性能。

ViTAR: 任意分辨率的视觉 Transformer

ViTAR: Vision Transformer with Any Resolution

Vision Transformers (ViTs) have achieved overwhelming success, yet they
suffer from vulnerable resolution scalability, i.e., the performance drops
drastically when presented with input resolutions that are unseen during
training. We introduce, ResFormer, a framework that is built upon the seminal
idea of multi-resolution training for improved performance on a wide spectrum
of, mostly unseen, testing resolutions. In particular, ResFormer operates on
replicated images of different resolutions and enforces a scale consistency
loss to engage interactive information across different scales. More
importantly, to alternate among varying resolutions effectively, especially
novel ones in testing, we propose a global-local positional embedding strategy
that changes smoothly conditioned on input sizes. We conduct extensive
experiments for image classification on ImageNet. The results provide strong
quantitative evidence that ResFormer has promising scaling abilities towards a
wide range of resolutions. For instance, ResFormer-B-MR achieves a Top-1
accuracy of 75.86% and 81.72% when evaluated on relatively low and high
resolutions respectively (i.e., 96 and 640), which are 48% and 7.49% better
than DeiT-B. We also demonstrate, moreover, ResFormer is flexible and can be
easily extended to semantic segmentation, object detection and video action
recognition. Code is available at this https URL

ResFormer 框架通过多分辨率训练方法和全局 - 局部位置嵌入策略，在图像分类、目标检测、语义分割和视频动作识别等领域取得了较好的结果。