Point-cloud data collected in real-world applications are often incomplete.
Data is typically missing due to objects being observed from partial
viewpoints, which only capture a specific perspective or angle. Additionally,
data can be incomplete due to occlusion and low-resolution sampling. Existing
completion approaches rely on datasets of predefined objects to guide the
completion of noisy and incomplete, point clouds. However, these approaches
perform poorly when tested on Out-Of-Distribution (OOD) objects, that are
poorly represented in the training dataset. Here we leverage recent advances in
text-guided image generation, which lead to major breakthroughs in text-guided
shape generation. We describe an approach called SDS-Complete that uses a
pre-trained text-to-image diffusion model and leverages the text semantics of a
given incomplete point cloud of an object, to obtain a complete surface
representation. SDS-Complete can complete a variety of objects using test-time
optimization without expensive collection of 3D information. We evaluate SDS
Complete on incomplete scanned objects, captured by real-world depth sensors
and LiDAR scanners. We find that it effectively reconstructs objects that are
absent from common datasets, reducing Chamfer loss by 50% on average compared
with current methods. Project page: this https URL

本文提出了一种名为 SDS-Complete 的方法，该方法利用预训练的文本到图像扩散模型，借助给定不完整点云对象的文本语义获取完整的表面表示，其可以有效地重建通过现实世界深度传感器和 LiDAR 扫描仪捕获的不完整扫描物体，相较于当前方法平均减少了 50％的 Chamfer 损失。