A well-known challenge in applying deep-learning methods to omnidirectional
images is spherical distortion. In dense regression tasks such as depth
estimation, where structural details are required, using a vanilla CNN layer on
the distorted 360 image results in undesired information loss. In this paper,
we propose a 360 monocular depth estimation pipeline, OmniFusion, to tackle the
spherical distortion issue. Our pipeline transforms a 360 image into
less-distorted perspective patches (i.e. tangent images) to obtain patch-wise
predictions via CNN, and then merge the patch-wise results for final output. To
handle the discrepancy between patch-wise predictions which is a major issue
affecting the merging quality, we propose a new framework with the following
key components. First, we propose a geometry-aware feature fusion mechanism
that combines 3D geometric features with 2D image features to compensate for
the patch-wise discrepancy. Second, we employ the self-attention-based
transformer architecture to conduct a global aggregation of patch-wise
information, which further improves the consistency. Last, we introduce an
iterative depth refinement mechanism, to further refine the estimated depth
based on the more accurate geometric features. Experiments show that our method
greatly mitigates the distortion issue, and achieves state-of-the-art
performances on several 360 monocular depth estimation benchmark datasets.

本篇论文提出了一种名为 OmniFusion 的 360 单目深度估计流程，通过将 360 图像转换为切线图像并采用 CNN 进行分段预测，最后合并分段结果以处理球形畸变问题，并通过几何感知特征融合机制、自注意力变换架构和迭代深度细化机制处理分段预测不一致问题，成功的在多个 360 单目深度估计基准数据集上取得了最先进的性能。