Depth estimation is a critical technology in autonomous driving, and
multi-camera systems are often used to achieve a 360{\deg} perception. These
360{\deg} camera sets often have limited or low-quality overlap regions, making
multi-view stereo methods infeasible for the entire image. Alternatively,
monocular methods may not produce consistent cross-view predictions. To address
these issues, we propose the Stereo Guided Depth Estimation (SGDE) method,
which enhances depth estimation of the full image by explicitly utilizing
multi-view stereo results on the overlap. We suggest building virtual pinhole
cameras to resolve the distortion problem of fisheye cameras and unify the
processing for the two types of 360{\deg} cameras. For handling the varying
noise on camera poses caused by unstable movement, the approach employs a
self-calibration method to obtain highly accurate relative poses of the
adjacent cameras with minor overlap. These enable the use of robust stereo
methods to obtain high-quality depth prior in the overlap region. This prior
serves not only as an additional input but also as pseudo-labels that enhance
the accuracy of depth estimation methods and improve cross-view prediction
consistency. The effectiveness of SGDE is evaluated on one fisheye camera
dataset, Synthetic Urban, and two pinhole camera datasets, DDAD and nuScenes.
Our experiments demonstrate that SGDE is effective for both supervised and
self-supervised depth estimation, and highlight the potential of our method for
advancing downstream autonomous driving technologies, such as 3D object
detection and occupancy prediction.

360 度摄像机的深度估计中，使用 Stereo Guided Depth Estimation (SGDE) 方法，通过利用重叠区域的多视图立体结果来提高整个图像的深度估计，解决了传统多视图立体方法和单目方法的局限性。

SDGE: 面向 360° 相机集合的立体引导深度估计

SDGE: Stereo Guided Depth Estimation for 360° Camera Sets

As 360{\deg} cameras become prevalent in many autonomous systems (e.g.,
self-driving cars and drones), efficient 360{\deg} perception becomes more and
more important. We propose a novel self-supervised learning approach for
predicting the omnidirectional depth and camera motion from a 360{\deg} video.
In particular, starting from the SfMLearner, which is designed for cameras with
normal field-of-view, we introduce three key features to process 360{\deg}
images efficiently. Firstly, we convert each image from equirectangular
projection to cubic projection in order to avoid image distortion. In each
network layer, we use Cube Padding (CP), which pads intermediate features from
adjacent faces, to avoid image boundaries. Secondly, we propose a novel
"spherical" photometric consistency constraint on the whole viewing sphere. In
this way, no pixel will be projected outside the image boundary which typically
happens in images with normal field-of-view. Finally, rather than naively
estimating six independent camera motions (i.e., naively applying SfM-Learner
to each face on a cube), we propose a novel camera pose consistency loss to
ensure the estimated camera motions reaching consensus. To train and evaluate
our approach, we collect a new PanoSUNCG dataset containing a large amount of
360{\deg} videos with groundtruth depth and camera motion. Our approach
achieves state-of-the-art depth prediction and camera motion estimation on
PanoSUNCG with faster inference speed comparing to equirectangular. In
real-world indoor videos, our approach can also achieve qualitatively
reasonable depth prediction by acquiring model pre-trained on PanoSUNCG.

本论文提出了一种新颖的自监督学习方法来从 360 度视频中预测全向深度和相机运动，通过转换图像类型，提出全视角光度一致性约束和相机姿态一致性误差等方法来提高视频运行速度，同时实现了比等面投影更快速的深度预测和相机运动估计。

360° 视频深度和相机运动的自监督学习

Self-Supervised Learning of Depth and Camera Motion from 360°  Videos

With the recent proliferation of consumer-grade 360{\deg} cameras, it is
worth revisiting visual perception challenges with spherical cameras given the
potential benefit of their global field of view. To this end we introduce a
spherical convolutional hourglass network (SCHN) for the dense labeling on the
sphere. The SCHN is invariant to camera orientation (lifting the usual
requirement for `upright' panoramic images), and its design is scalable for
larger practical datasets. Initial experiments show promising results on a
spherical semantic segmentation task.

本文介绍一种球面卷积沙漏网络（SCHN）并针对全景照相机中的视觉感知挑战进行研究，该网络可以进行密集标注和球面语义分割，并针对大型实际数据集进行可扩展的设计，初步实验表明该方法是有希望的。