We propose a novel zero-shot approach to computing correspondences between 3D
shapes. Existing approaches mainly focus on isometric and near-isometric shape
pairs (e.g., human vs. human), but less attention has been given to strongly
non-isometric and inter-class shape matching (e.g., human vs. cow). To this
end, we introduce a fully automatic method that exploits the exceptional
reasoning capabilities of recent foundation models in language and vision to
tackle difficult shape correspondence problems. Our approach comprises multiple
stages. First, we classify the 3D shapes in a zero-shot manner by feeding
rendered shape views to a language-vision model (e.g., BLIP2) to generate a
list of class proposals per shape. These proposals are unified into a single
class per shape by employing the reasoning capabilities of ChatGPT. Second, we
attempt to segment the two shapes in a zero-shot manner, but in contrast to the
co-segmentation problem, we do not require a mutual set of semantic regions.
Instead, we propose to exploit the in-context learning capabilities of ChatGPT
to generate two different sets of semantic regions for each shape and a
semantic mapping between them. This enables our approach to match strongly
non-isometric shapes with significant differences in geometric structure.
Finally, we employ the generated semantic mapping to produce coarse
correspondences that can further be refined by the functional maps framework to
produce dense point-to-point maps. Our approach, despite its simplicity,
produces highly plausible results in a zero-shot manner, especially between
strongly non-isometric shapes.

本文提出一种新颖的零样本方法，用于计算 3D 模型之间的对应关系，特别是针对具有很强差异性和不同类别之间匹配的问题，并在零样本情况下使用 language-vision model 方法进行分类，使用 ChatGPT 生成语义映射，并使用 functional maps framework 进一步细化到点对点映射。

零射 3D 形状对应

Zero-Shot 3D Shape Correspondence

We propose a method for efficiently computing orientation-preserving and
approximately continuous correspondences between non-rigid shapes, using the
functional maps framework. We first show how orientation preservation can be
formulated directly in the functional (spectral) domain without using landmark
or region correspondences and without relying on external symmetry information.
This allows us to obtain functional maps that promote orientation preservation,
even when using descriptors, that are invariant to orientation changes. We then
show how higher quality, approximately continuous and bijective pointwise
correspondences can be obtained from initial functional maps by introducing a
novel refinement technique that aims to simultaneously improve the maps both in
the spectral and spatial domains. This leads to a general pipeline for
computing correspondences between shapes that results in high-quality maps,
while admitting an efficient optimization scheme. We show through extensive
evaluation that our approach improves upon state-of-the-art results on
challenging isometric and non-isometric correspondence benchmarks according to
both measures of continuity and coverage as well as producing semantically
meaningful correspondences as measured by the distance to ground truth maps.

本文提出了一种方法，利用函数映射框架高效计算非刚性形状之间具有保方向和近似连续性的对应关系，并通过引入新的方法对初始函数映射进行改进，从而获得高质量的点对应关系。研究表明，该方法在保持连续性和覆盖率方面显著优于现有技术，并且产生的对应关系是有意义的。