With the development of the neural field, reconstructing the 3D model of a
target object from multi-view inputs has recently attracted increasing
attention from the community. Existing methods normally learn a neural field
for the whole scene, while it is still under-explored how to reconstruct a
certain object indicated by users on-the-fly. Considering the Segment Anything
Model (SAM) has shown effectiveness in segmenting any 2D images, in this paper,
we propose Neural Object Cloning (NOC), a novel high-quality 3D object
reconstruction method, which leverages the benefits of both neural field and
SAM from two aspects. Firstly, to separate the target object from the scene, we
propose a novel strategy to lift the multi-view 2D segmentation masks of SAM
into a unified 3D variation field. The 3D variation field is then projected
into 2D space and generates the new prompts for SAM. This process is iterative
until convergence to separate the target object from the scene. Then, apart
from 2D masks, we further lift the 2D features of the SAM encoder into a 3D SAM
field in order to improve the reconstruction quality of the target object. NOC
lifts the 2D masks and features of SAM into the 3D neural field for
high-quality target object reconstruction. We conduct detailed experiments on
several benchmark datasets to demonstrate the advantages of our method. The
code will be released.

通过将 Segment Anything Model（SAM）中的 2D 分割掩膜传递到统一的 3D 变化场中，然后进一步将 SAM 编码器的 2D 特征提取到 3D SAM 场中，提出了 Neural Object Cloning（NOC）方法来高质量地重建目标对象的 3D 模型。