Existing panoramic depth estimation methods based on convolutional neural
networks (CNNs) focus on removing panoramic distortions, failing to perceive
panoramic structures efficiently due to the fixed receptive field in CNNs. This
paper proposes the panorama transformer (named PanoFormer) to estimate the
depth in panorama images, with tangent patches from spherical domain, learnable
token flows, and panorama specific metrics. In particular, we divide patches on
the spherical tangent domain into tokens to reduce the negative effect of
panoramic distortions. Since the geometric structures are essential for depth
estimation, a self-attention module is redesigned with an additional learnable
token flow. In addition, considering the characteristic of the spherical
domain, we present two panorama-specific metrics to comprehensively evaluate
the panoramic depth estimation models' performance. Extensive experiments
demonstrate that our approach significantly outperforms the state-of-the-art
(SOTA) methods. Furthermore, the proposed method can be effectively extended to
solve semantic panorama segmentation, a similar pixel2pixel task. Code will be
available.

本文提出了一种名为 PanoFormer 的全景深度估计方法，通过使用来自球形域的切向补丁、可学习的 token 流以及全景特定指标来提高全景结构的感知能力，并通过重新设计自注意力模块来增强其拟合全景几何结构的能力，经实验证明其取得了比现有的 SOTA 方法更好的表现，并且可以有效地扩展以解决语义全景分割问题。