Most existing ultra-high resolution (UHR) segmentation methods always
struggle in the dilemma of balancing memory cost and local characterization
accuracy, which are both taken into account in our proposed Guided
Patch-Grouping Wavelet Transformer (GPWFormer) that achieves impressive
performances. In this work, GPWFormer is a Transformer ($\mathcal{T}$)-CNN
($\mathcal{C}$) mutual leaning framework, where $\mathcal{T}$ takes the whole
UHR image as input and harvests both local details and fine-grained long-range
contextual dependencies, while $\mathcal{C}$ takes downsampled image as input
for learning the category-wise deep context. For the sake of high inference
speed and low computation complexity, $\mathcal{T}$ partitions the original UHR
image into patches and groups them dynamically, then learns the low-level local
details with the lightweight multi-head Wavelet Transformer (WFormer) network.
Meanwhile, the fine-grained long-range contextual dependencies are also
captured during this process, since patches that are far away in the spatial
domain can also be assigned to the same group. In addition, masks produced by
$\mathcal{C}$ are utilized to guide the patch grouping process, providing a
heuristics decision. Moreover, the congruence constraints between the two
branches are also exploited to maintain the spatial consistency among the
patches. Overall, we stack the multi-stage process in a pyramid way.
Experiments show that GPWFormer outperforms the existing methods with
significant improvements on five benchmark datasets.

本文提出了一种基于 Guided Patch-Grouping Wavelet Transformer 的 Transformer-CNN 相互学习框架，其中通过将原始 UHR 图像划分为补丁并在动态分组中对其进行轻量级多头小波变换器（WFormer）网络的学习来学习低级别本地细节，同时也可以捕获这个过程中的细粒度长距离上下文依赖，并利用 CNN 生成的掩码来指导贴图分组过程，提供一种启发式决策；同时利用两个分支之间的一致性约束来维护图像块之间的空间一致性，实验结果表明该方法在五个基准数据集上展现出较好的性能。