As 360{\deg} cameras become prevalent in many autonomous systems (e.g.,
self-driving cars and drones), efficient 360{\deg} perception becomes more and
more important. We propose a novel self-supervised learning approach for
predicting the omnidirectional depth and camera motion from a 360{\deg} video.
In particular, starting from the SfMLearner, which is designed for cameras with
normal field-of-view, we introduce three key features to process 360{\deg}
images efficiently. Firstly, we convert each image from equirectangular
projection to cubic projection in order to avoid image distortion. In each
network layer, we use Cube Padding (CP), which pads intermediate features from
adjacent faces, to avoid image boundaries. Secondly, we propose a novel
"spherical" photometric consistency constraint on the whole viewing sphere. In
this way, no pixel will be projected outside the image boundary which typically
happens in images with normal field-of-view. Finally, rather than naively
estimating six independent camera motions (i.e., naively applying SfM-Learner
to each face on a cube), we propose a novel camera pose consistency loss to
ensure the estimated camera motions reaching consensus. To train and evaluate
our approach, we collect a new PanoSUNCG dataset containing a large amount of
360{\deg} videos with groundtruth depth and camera motion. Our approach
achieves state-of-the-art depth prediction and camera motion estimation on
PanoSUNCG with faster inference speed comparing to equirectangular. In
real-world indoor videos, our approach can also achieve qualitatively
reasonable depth prediction by acquiring model pre-trained on PanoSUNCG.

本论文提出了一种新颖的自监督学习方法来从 360 度视频中预测全向深度和相机运动，通过转换图像类型，提出全视角光度一致性约束和相机姿态一致性误差等方法来提高视频运行速度，同时实现了比等面投影更快速的深度预测和相机运动估计。