We present ShapeClipper, a novel method that reconstructs 3D object shapes
from real-world single-view RGB images. Instead of relying on laborious 3D,
multi-view or camera pose annotation, ShapeClipper learns shape reconstruction
from a set of single-view segmented images. The key idea is to facilitate shape
learning via CLIP-based shape consistency, where we encourage objects with
similar CLIP encodings to share similar shapes. We also leverage off-the-shelf
normals as an additional geometric constraint so the model can learn better
bottom-up reasoning of detailed surface geometry. These two novel consistency
constraints, when used to regularize our model, improve its ability to learn
both global shape structure and local geometric details. We evaluate our method
over three challenging real-world datasets, Pix3D, Pascal3D+, and OpenImages,
where we achieve superior performance over state-of-the-art methods.

ShapeClipper 是一种利用单视角图像实现三维物体形状重建的新方法，其中采用基于 CLIP 的形状一致性来促进物体的形状学习，并利用现成的法线作为额外的几何约束，以实现对详细表面几何结构的自下而上推理。我们在 Pix3D、Pascal3D + 和 OpenImages 等三个数据集上进行了评估，并取得了比最先进方法更好的性能。

ShapeClipper：基于几何和 CLIP 一致性的单视图图像可扩展的三维形状学习

ShapeClipper: Scalable 3D Shape Learning from Single-View Images via  Geometric and CLIP-based Consistency

We present a framework for learning 3D object shapes and dense cross-object
3D correspondences from just an unaligned category-specific image collection.
The 3D shapes are generated implicitly as deformations to a category-specific
signed distance field and are learned in an unsupervised manner solely from
unaligned image collections and their poses without any 3D supervision.
Generally, image collections on the internet contain several intra-category
geometric and topological variations, for example, different chairs can have
different topologies, which makes the task of joint shape and correspondence
estimation much more challenging. Because of this, prior works either focus on
learning each 3D object shape individually without modeling cross-instance
correspondences or perform joint shape and correspondence estimation on
categories with minimal intra-category topological variations. We overcome
these restrictions by learning a topologically-aware implicit deformation field
that maps a 3D point in the object space to a higher dimensional point in the
category-specific canonical space. At inference time, given a single image, we
reconstruct the underlying 3D shape by first implicitly deforming each 3D point
in the object space to the learned category-specific canonical space using the
topologically-aware deformation field and then reconstructing the 3D shape as a
canonical signed distance field. Both canonical shape and deformation field are
learned end-to-end in an inverse-graphics fashion using a learned recurrent ray
marcher (SRN) as a differentiable rendering module. Our approach, dubbed TARS,
achieves state-of-the-art reconstruction fidelity on several datasets:
ShapeNet, Pascal3D+, CUB, and Pix3D chairs. Result videos and code at
this https URL

无需对齐的图像集合，基于拓扑意识的隐式形变场的学习、随后进行形状重构，该方法名为 TARS，在多个数据集中均取得了最先进的重构精度 (ShapeNet, Pascal3D+, CUB, and Pix3D chairs)。