We introduce an active 3D reconstruction method which integrates visual
perception, robot-object interaction, and 3D scanning to recover both the
exterior and interior, i.e., unexposed, geometries of a target 3D object.
Unlike other works in active vision which focus on optimizing camera viewpoints
to better investigate the environment, the primary feature of our
reconstruction is an analysis of the interactability of various parts of the
target object and the ensuing part manipulation by a robot to enable scanning
of occluded regions. As a result, an understanding of part articulations of the
target object is obtained on top of complete geometry acquisition. Our method
operates fully automatically by a Fetch robot with built-in RGBD sensors. It
iterates between interaction analysis and interaction-driven reconstruction,
scanning and reconstructing detected moveable parts one at a time, where both
the articulated part detection and mesh reconstruction are carried out by
neural networks. In the final step, all the remaining, non-articulated parts,
including all the interior structures that had been exposed by prior part
manipulations and subsequently scanned, are reconstructed to complete the
acquisition. We demonstrate the performance of our method via qualitative and
quantitative evaluation, ablation studies, comparisons to alternatives, as well
as experiments in a real environment.

利用感知、机器人对象交互和三维扫描的积极三维重建方法，恢复目标三维物体的外部和内部几何结构，通过分析各部分的互动性和机器人的部分操作来扫描遮挡区域，自动化操作通过内嵌 RGBD 传感器的 Fetch 机器人完成，最后重建所有未暴露的非活动部分和内部结构完成获取。

基于交互的带有物体内部的主动三维重建

Interaction-Driven Active 3D Reconstruction with Object Interiors

We present an approach to learn an object-centric forward model, and show
that this allows us to plan for sequences of actions to achieve distant desired
goals. We propose to model a scene as a collection of objects, each with an
explicit spatial location and implicit visual feature, and learn to model the
effects of actions using random interaction data. Our model allows capturing
the robot-object and object-object interactions, and leads to more
sample-efficient and accurate predictions. We show that this learned model can
be leveraged to search for action sequences that lead to desired goal
configurations, and that in conjunction with a learned correction module, this
allows for robust closed loop execution. We present experiments both in
simulation and the real world, and show that our approach improves over
alternate implicit or pixel-space forward models. Please see our project page
(this https URL) for result videos.

该论文提出了一种基于物体的前向模型学习方法，用于规划动作序列以实现远程期望目标，并通过随机交互数据来学习行为效果。实验结果表明，该方法具有更高的采样效率和准确性，可以优化闭环执行。