We introduce a method to convert stereo 360{\deg} (omnidirectional stereo) imagery into a layered, multi-sphere image representation for six degree-of-freedom (6DoF) rendering. Stereo 360{\deg} imagery can be captured from multi-camera systems for virtual reality (VR), but lacks motion parallax and correct-in-all-directions disparity cues. Together, these can quickly lead to VR sickness when viewing content. One solution is to try and generate a format suitable for 6DoF rendering, such as by estimating depth. However, this raises questions as to how to handle disoccluded regions in dynamic scenes. Our approach is to simultaneously learn depth and disocclusions via a multi-sphere image representation, which can be rendered with correct 6DoF disparity and motion parallax in VR. This significantly improves comfort for the viewer, and can be inferred and rendered in real time on modern GPU hardware. Together, these move towards making VR video a more comfortable immersive medium.

本文介绍了一种将立体360度（全向立体）成像转换为分层式多球形图像表示的方法，用于六自由度（6DoF）渲染。该方法利用多摄像机系统捕获立体360度图像，同时解决其中缺少动态场景的运动视差和正确各项异性差的问题，实现舒适的VR体验。

MatryODShka: 使用多球体图像实时生成6自由度视图合成视频