Estimating the accurate depth from a single image is challenging since it is
inherently ambiguous and ill-posed. While recent works design increasingly
complicated and powerful networks to directly regress the depth map, we take
the path of CRFs optimization. Due to the expensive computation, CRFs are
usually performed between neighborhoods rather than the whole graph. To
leverage the potential of fully-connected CRFs, we split the input into windows
and perform the FC-CRFs optimization within each window, which reduces the
computation complexity and makes FC-CRFs feasible. To better capture the
relationships between nodes in the graph, we exploit the multi-head attention
mechanism to compute a multi-head potential function, which is fed to the
networks to output an optimized depth map. Then we build a bottom-up-top-down
structure, where this neural window FC-CRFs module serves as the decoder, and a
vision transformer serves as the encoder. The experiments demonstrate that our
method significantly improves the performance across all metrics on both the
KITTI and NYUv2 datasets, compared to previous methods. Furthermore, the
proposed method can be directly applied to panorama images and outperforms all
previous panorama methods on the MatterPort3D dataset. Project page:
this https URL

本研究提出了一种使用 CRFs 优化和多头注意机制的深度估计方法，其中神经窗口 FC-CRFs 模块作为解码器，视觉变换器作为编码器，结果表明相对于以前的方法，在 KITTI 和 NYUv2 数据集上均能显著提高性能，并且该方法可直接应用于全景图像。