We present recurrent geometry-aware neural networks that integrate visual
information across multiple views of a scene into 3D latent feature tensors,
while maintaining an one-to-one mapping between 3D physical locations in the
world scene and latent feature locations. Object detection, object
segmentation, and 3D reconstruction is then carried out directly using the
constructed 3D feature memory, as opposed to any of the input 2D images. The
proposed models are equipped with differentiable egomotion-aware feature
warping and (learned) depth-aware unprojection operations to achieve
geometrically consistent mapping between the features in the input frame and
the constructed latent model of the scene. We empirically show the proposed
model generalizes much better than geometryunaware LSTM/GRU networks,
especially under the presence of multiple objects and cross-object occlusions.
Combined with active view selection policies, our model learns to select
informative viewpoints to integrate information from by "undoing" cross-object
occlusions, seamlessly combining geometry with learning from experience.

该研究介绍了使用递归的几何感知神经网络，将一个场景的多个视图中的视觉信息整合到 3D 潜在特征张量中，并直接使用构建的 3D 特征存储器进行物体检测、物体分割和 3D 重建。