Panoramic images with their 360-degree directional view encompass exhaustive
information about the surrounding space, providing a rich foundation for scene
understanding. To unfold this potential in the form of robust panoramic
segmentation models, large quantities of expensive, pixel-wise annotations are
crucial for success. Such annotations are available, but predominantly for
narrow-angle, pinhole-camera images which, off the shelf, serve as sub-optimal
resources for training panoramic models. Distortions and the distinct
image-feature distribution in 360-degree panoramas impede the transfer from the
annotation-rich pinhole domain and therefore come with a big dent in
performance. To get around this domain difference and bring together semantic
annotations from pinhole- and 360-degree surround-visuals, we propose to learn
object deformations and panoramic image distortions in the Deformable Patch
Embedding (DPE) and Deformable MLP (DMLP) components which blend into our
Transformer for PAnoramic Semantic Segmentation (Trans4PASS) model. Finally, we
tie together shared semantics in pinhole- and panoramic feature embeddings by
generating multi-scale prototype features and aligning them in our Mutual
Prototypical Adaptation (MPA) for unsupervised domain adaptation. On the indoor
Stanford2D3D dataset, our Trans4PASS with MPA maintains comparable performance
to fully-supervised state-of-the-arts, cutting the need for over 1,400 labeled
panoramas. On the outdoor DensePASS dataset, we break state-of-the-art by
14.39% mIoU and set the new bar at 56.38%. Code will be made publicly available
at this https URL

利用变形补丁嵌入（DPE）和变形 MLP（DMLP），并结合传输学习方法，将针孔相机图像的语义注释和 360 度环视视觉的语义注释结合起来，从而实现全景图像的稳健分割，并通过互补原型自适应实现无监督域自适应。在 Stanford2D3D 数据集中，与完全受监督的现有技术相比，我们的 Trans4PASS 与 MPA 保持可比的性能，而无需超过 1,400 个标记化全景图像。在室外 DensePASS 数据集中，我们打破了现有技术水平，使 mIoU 提高了 14.39％，将新标准设定为 56.38％。