We introduce a method that simultaneously learns to explore new large
environments and to reconstruct them in 3D from color images only. This is
closely related to the Next Best View problem (NBV), where one has to identify
where to move the camera next to improve the coverage of an unknown scene.
However, most of the current NBV methods rely on depth sensors, need 3D
supervision and/or do not scale to large scenes. Our method requires only a
color camera and no 3D supervision. It simultaneously learns in a
self-supervised fashion to predict a "volume occupancy field" from color images
and, from this field, to predict the NBV. Thanks to this approach, our method
performs well on new scenes as it is not biased towards any training 3D data.
We demonstrate this on a recent dataset made of various 3D scenes and show it
performs even better than recent methods requiring a depth sensor, which is not
a realistic assumption for outdoor scenes captured with a flying drone.

本研究提出了一种无需使用深度传感器，只需使用彩色相机即可同时学习在大型环境中探索和进行 3D 重建的方法，并基于此领域预测下一个最佳视图。与现有的方法相比，我们的方法无需 3D 监督，通过进行自监督学习可以在新场景上表现出很好的性能。