We present En3D, an enhanced generative scheme for sculpting high-quality 3D
human avatars. Unlike previous works that rely on scarce 3D datasets or limited
2D collections with imbalanced viewing angles and imprecise pose priors, our
approach aims to develop a zero-shot 3D generative scheme capable of producing
visually realistic, geometrically accurate and content-wise diverse 3D humans
without relying on pre-existing 3D or 2D assets. To address this challenge, we
introduce a meticulously crafted workflow that implements accurate physical
modeling to learn the enhanced 3D generative model from synthetic 2D data.
During inference, we integrate optimization modules to bridge the gap between
realistic appearances and coarse 3D shapes. Specifically, En3D comprises three
modules: a 3D generator that accurately models generalizable 3D humans with
realistic appearance from synthesized balanced, diverse, and structured human
images; a geometry sculptor that enhances shape quality using multi-view normal
constraints for intricate human anatomy; and a texturing module that
disentangles explicit texture maps with fidelity and editability, leveraging
semantical UV partitioning and a differentiable rasterizer. Experimental
results show that our approach significantly outperforms prior works in terms
of image quality, geometry accuracy and content diversity. We also showcase the
applicability of our generated avatars for animation and editing, as well as
the scalability of our approach for content-style free adaptation.

通过以人为基础的准确物理建模，我们提出了一种增强的生成方案 En3D，能够从合成的平衡、多样化和结构化的人类图像中，准确地模拟具有逼真外观的通用化 3D 人物。

En3D：从 2D 合成数据雕刻 3D 人体的增强生成模型

En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D  Synthetic Data

Can we synthesize 3D humans interacting with scenes without learning from any
3D human-scene interaction data? We propose GenZI, the first zero-shot approach
to generating 3D human-scene interactions. Key to GenZI is our distillation of
interaction priors from large vision-language models (VLMs), which have learned
a rich semantic space of 2D human-scene compositions. Given a natural language
description and a coarse point location of the desired interaction in a 3D
scene, we first leverage VLMs to imagine plausible 2D human interactions
inpainted into multiple rendered views of the scene. We then formulate a robust
iterative optimization to synthesize the pose and shape of a 3D human model in
the scene, guided by consistency with the 2D interaction hypotheses. In
contrast to existing learning-based approaches, GenZI circumvents the
conventional need for captured 3D interaction data, and allows for flexible
control of the 3D interaction synthesis with easy-to-use text prompts.
Extensive experiments show that our zero-shot approach has high flexibility and
generality, making it applicable to diverse scene types, including both indoor
and outdoor environments.

可以不借助任何 3D 人 - 场景交互数据合成 3D 人与场景交互吗？我们提出了 GenZI，这是第一个零样本方法，用于生成 3D 人与场景的交互。GenZI 的关键在于从大型视觉 - 语言模型 (VLMs) 中提取交互先验知识，这些先验知识学习了丰富的二维人 - 场景组合的语义空间。通过给定自然语言描述和 3D 场景中所需交互的粗略点位置，我们首先利用 VLMs 来想象描绘在场景的多个渲染视图中的可信的二维人交互。然后，我们通过与 2D 交互假设的一致性引导，制定一个鲁棒的迭代优化过程，合成场景中的 3D 人模型的姿态和形状。与现有的基于学习的方法相比，GenZI 避免了传统上对捕获的 3D 交互数据的需求，并允许使用简单易用的文字提示对 3D 交互合成进行灵活控制。大量实验证明我们的零样本方法具有高灵活性和广泛适用性，可适用于包括室内和室外环境在内的各种场景类型。

GenZI: 零 Shot 3D 人物场景交互生成

GenZI: Zero-Shot 3D Human-Scene Interaction Generation

Most prior works in perceiving 3D humans from images reason human in
isolation without their surroundings. However, humans are constantly
interacting with the surrounding objects, thus calling for models that can
reason about not only the human but also the object and their interaction. The
problem is extremely challenging due to heavy occlusions between humans and
objects, diverse interaction types and depth ambiguity. In this paper, we
introduce CHORE, a novel method that learns to jointly reconstruct the human
and the object from a single RGB image. CHORE takes inspiration from recent
advances in implicit surface learning and classical model-based fitting. We
compute a neural reconstruction of human and object represented implicitly with
two unsigned distance fields, a correspondence field to a parametric body and
an object pose field. This allows us to robustly fit a parametric body model
and a 3D object template, while reasoning about interactions. Furthermore,
prior pixel-aligned implicit learning methods use synthetic data and make
assumptions that are not met in the real data. We propose a elegant depth-aware
scaling that allows more efficient shape learning on real data. Experiments
show that our joint reconstruction learned with the proposed strategy
significantly outperforms the SOTA. Our code and models are available at
this https URL

该研究使用一种名为 CHORE 的新方法，通过学习从单个 RGB 图像中重建人和物体的神经重建来解决人和周围物体之间的交互问题，而不是像以往那样只考虑人与周围环境的孤立问题，并通过对深度进行合理的缩放操作，更高效地处理真实数据，实验结果证明相较于已有最佳方法 SOTA，该方法的联合重建表现更好。