Multi-scale architecture, including hierarchical vision transformer, has been
commonly applied to high-resolution semantic segmentation to deal with
computational complexity with minimum performance loss. In this paper, we
propose a novel decoding scheme for semantic segmentation in this regard, which
takes multi-level features from the encoder with multi-scale architecture. The
decoding scheme based on a multi-level vision transformer aims to achieve not
only reduced computational expense but also higher segmentation accuracy, by
introducing successive cross-attention in aggregation of the multi-level
features. Furthermore, a way to enhance the multi-level features by the
aggregated semantics is proposed. The effort is focused on maintaining the
contextual consistency from the perspective of attention allocation and brings
improved performance with significantly lower computational cost. Set of
experiments on popular datasets demonstrates superiority of the proposed scheme
to the state-of-the-art semantic segmentation models in terms of computational
cost without loss of accuracy, and extensive ablation studies prove the
effectiveness of ideas proposed.

通过多级视觉变换编码器中的多级特征，本文提出了一种新颖的语义分割解码方案，旨在降低计算开销、提高分割准确性，并通过聚合后的语义增强多级特征，从注意力分配的角度保持上下文一致性，大幅降低计算成本。实验结果表明该方案在计算成本上优于现有语义分割模型，且没有准确性损失；深入的消融研究证明了所提出思路的有效性。

轻量级语义分割的多层特征逐级聚合解码方案

A Decoding Scheme with Successive Aggregation of Multi-Level Features  for Light-Weight Semantic Segmentation

Medical image segmentation has seen significant improvements with transformer
models, which excel in grasping far-reaching contexts and global contextual
information. However, the increasing computational demands of these models,
proportional to the squared token count, limit their depth and resolution
capabilities. Most current methods process D volumetric image data
slice-by-slice (called pseudo 3D), missing crucial inter-slice information and
thus reducing the model's overall performance. To address these challenges, we
introduce the concept of \textbf{Deformable Large Kernel Attention (D-LKA
Attention)}, a streamlined attention mechanism employing large convolution
kernels to fully appreciate volumetric context. This mechanism operates within
a receptive field akin to self-attention while sidestepping the computational
overhead. Additionally, our proposed attention mechanism benefits from
deformable convolutions to flexibly warp the sampling grid, enabling the model
to adapt appropriately to diverse data patterns. We designed both 2D and 3D
adaptations of the D-LKA Attention, with the latter excelling in cross-depth
data understanding. Together, these components shape our novel hierarchical
Vision Transformer architecture, the \textit{D-LKA Net}. Evaluations of our
model against leading methods on popular medical segmentation datasets
(Synapse, NIH Pancreas, and Skin lesion) demonstrate its superior performance.
Our code implementation is publicly available at the:
this https URL

介绍了一种新的分层视觉 Transformer 网络模型 (D-LKA Net)，采用变形大核注意力机制以完全理解体积上下文信息，并在医学图像分割任务上展现出卓越的性能。

超越自注意力：用于医学图像分割的可变形大卷积核注意力

Beyond Self-Attention: Deformable Large Kernel Attention for Medical  Image Segmentation

Transformers are quickly becoming one of the most heavily applied deep
learning architectures across modalities, domains, and tasks. In vision, on top
of ongoing efforts into plain transformers, hierarchical transformers have also
gained significant attention, thanks to their performance and easy integration
into existing frameworks. These models typically employ localized attention
mechanisms, such as the sliding-window Neighborhood Attention (NA) or Swin
Transformer's Shifted Window Self Attention. While effective at reducing self
attention's quadratic complexity, local attention weakens two of the most
desirable properties of self attention: long range inter-dependency modeling,
and global receptive field. In this paper, we introduce Dilated Neighborhood
Attention (DiNA), a natural, flexible and efficient extension to NA that can
capture more global context and expand receptive fields exponentially at no
additional cost. NA's local attention and DiNA's sparse global attention
complement each other, and therefore we introduce Dilated Neighborhood
Attention Transformer (DiNAT), a new hierarchical vision transformer built upon
both. DiNAT variants enjoy significant improvements over strong baselines such
as NAT, Swin, and ConvNeXt. Our large model is faster and ahead of its Swin
counterpart by 1.6% box AP in COCO object detection, 1.4% mask AP in COCO
instance segmentation, and 1.4% mIoU in ADE20K semantic segmentation. Paired
with new frameworks, our large variant is the new state of the art panoptic
segmentation model on COCO (58.5 PQ) and ADE20K (49.4 PQ), and instance
segmentation model on Cityscapes (45.1 AP) and ADE20K (35.4 AP) (no extra
data). It also matches the state of the art specialized semantic segmentation
models on ADE20K (58.1 mIoU), and ranks second on Cityscapes (84.5 mIoU) (no
extra data).

本文介绍了一种新型 Hierarchical Vision Transformer，Dilated Neighborhood Attention Transformer（DiNAT）及其基于 Dilated Neighborhood Attention（DiNA）的自注意力机制，使其具备全局关联建模和扩大感受野的能力，证明 DiNAT 对常规 Hierarchical Vision Transformer 具有显著的优势，从而实现了在一些特定领域中更好的 Panoptic 分割表现。

扩张邻域注意力变换器

Dilated Neighborhood Attention Transformer

Vision-and-language navigation (VLN) aims to build autonomous visual agents
that follow instructions and navigate in real scenes. To remember previously
visited locations and actions taken, most approaches to VLN implement memory
using recurrent states. Instead, we introduce a History Aware Multimodal
Transformer (HAMT) to incorporate a long-horizon history into multimodal
decision making. HAMT efficiently encodes all the past panoramic observations
via a hierarchical vision transformer (ViT), which first encodes individual
images with ViT, then models spatial relation between images in a panoramic
observation and finally takes into account temporal relation between panoramas
in the history. It, then, jointly combines text, history and current
observation to predict the next action. We first train HAMT end-to-end using
several proxy tasks including single step action prediction and spatial
relation prediction, and then use reinforcement learning to further improve the
navigation policy. HAMT achieves new state of the art on a broad range of VLN
tasks, including VLN with fine-grained instructions (R2R, RxR), high-level
instructions (R2R-Last, REVERIE), dialogs (CVDN) as well as long-horizon VLN
(R4R, R2R-Back). We demonstrate HAMT to be particularly effective for
navigation tasks with longer trajectories.

通过引入一种历史感知的多模式转换器来将长期历史包含在多模式决策中，以在视觉和语言导航中构建自主的视觉代理，进而通过使用强化学习进一步优化导航策略。