Given a new $6DoF$ camera pose in an indoor environment, we study the
challenging problem of predicting the view from that pose based on a set of
reference RGBD views. Existing explicit or implicit 3D geometry construction
methods are computationally expensive while those based on learning have
predominantly focused on isolated views of object categories with regular
geometric structure. Differing from the traditional \textit{render-inpaint}
approach to new view synthesis in the real indoor environment, we propose a
conditional generative adversarial neural network (P2I-NET) to directly predict
the new view from the given pose. P2I-NET learns the conditional distribution
of the images of the environment for establishing the correspondence between
the camera pose and its view of the environment, and achieves this through a
number of innovative designs in its architecture and training lost function.
Two auxiliary discriminator constraints are introduced for enforcing the
consistency between the pose of the generated image and that of the
corresponding real world image in both the latent feature space and the real
world pose space. Additionally a deep convolutional neural network (CNN) is
introduced to further reinforce this consistency in the pixel space. We have
performed extensive new view synthesis experiments on real indoor datasets.
Results show that P2I-NET has superior performance against a number of NeRF
based strong baseline models. In particular, we show that P2I-NET is 40 to 100
times faster than these competitor techniques while synthesising similar
quality images. Furthermore, we contribute a new publicly available indoor
environment dataset containing 22 high resolution RGBD videos where each frame
also has accurate camera pose parameters.

基于给定的室内环境中的新 6DoF 相机位姿，研究基于一组参考 RGBD 视图预测该位姿下的视角的难题，并提出了一种条件生成对抗神经网络（P2I-NET）来直接预测给定位姿下的新视角。