Despite recent research advancements in reconstructing clothed humans from a
single image, accurately restoring the "unseen regions" with high-level details
remains an unsolved challenge that lacks attention. Existing methods often
generate overly smooth back-side surfaces with a blurry texture. But how to
effectively capture all visual attributes of an individual from a single image,
which are sufficient to reconstruct unseen areas (e.g., the back view)?
Motivated by the power of foundation models, TeCH reconstructs the 3D human by
leveraging 1) descriptive text prompts (e.g., garments, colors, hairstyles)
which are automatically generated via a garment parsing model and Visual
Question Answering (VQA), 2) a personalized fine-tuned Text-to-Image diffusion
model (T2I) which learns the "indescribable" appearance. To represent
high-resolution 3D clothed humans at an affordable cost, we propose a hybrid 3D
representation based on DMTet, which consists of an explicit body shape grid
and an implicit distance field. Guided by the descriptive prompts +
personalized T2I diffusion model, the geometry and texture of the 3D humans are
optimized through multi-view Score Distillation Sampling (SDS) and
reconstruction losses based on the original observation. TeCH produces
high-fidelity 3D clothed humans with consistent & delicate texture, and
detailed full-body geometry. Quantitative and qualitative experiments
demonstrate that TeCH outperforms the state-of-the-art methods in terms of
reconstruction accuracy and rendering quality. The code will be publicly
available for research purposes at this https URL

通过使用描述性文本启示和个性化模型，TeCH 提出了一种混合的三维重建方法，用于重建细节丰富的衣着人物的未见区域，并在重建准确性和渲染质量方面优于现有方法。

TeCH: 文本引导下的逼真着装人体重建

TeCH: Text-guided Reconstruction of Lifelike Clothed Humans

We present a novel method for reconstructing clothed humans from a sparse set
of, e.g., 1 to 6 RGB images. Despite impressive results from recent works
employing deep implicit representation, we revisit the volumetric approach and
demonstrate that better performance can be achieved with proper system design.
The volumetric representation offers significant advantages in leveraging 3D
spatial context through 3D convolutions, and the notorious quantization error
is largely negligible with a reasonably large yet affordable volume resolution,
e.g., 512. To handle memory and computation costs, we propose a sophisticated
coarse-to-fine strategy with voxel culling and subspace sparse convolution. Our
method starts with a discretized visual hull to compute a coarse shape and then
focuses on a narrow band nearby the coarse shape for refinement. Once the shape
is reconstructed, we adopt an image-based rendering approach, which computes
the colors of surface points by blending input images with learned weights.
Extensive experimental results show that our method significantly reduces the
mean point-to-surface (P2S) precision of state-of-the-art methods by more than
50% to achieve approximately 2mm accuracy with a 512 volume resolution.
Additionally, images rendered from our textured model achieve a higher peak
signal-to-noise ratio (PSNR) compared to state-of-the-art methods.

我们提出了一种新颖的方法，用于从少量的 RGB 图像（例如 1 到 6 张）中重建穿着衣物的人类。我们重新审视了体素方法并证明了通过适当的系统设计可以获得更好的性能。体积表示通过 3D 卷积在利用三维空间上下文方面具有明显优势，对于合理的大型但保守的体积分辨率（例如 512），明显的量化误差可以忽略不计。为了处理内存和计算成本，我们提出了一个复杂的由粗到精的策略，包括体素剔除和子空间稀疏卷积。我们的方法从离散的视觉外壳开始计算粗糙形状，然后专注于粗糙形状附近的窄带进行细化。形状重建后，我们采用基于图像的渲染方法，通过将输入图像与学习到的权重混合来计算表面点的颜色。大量的实验结果表明，我们的方法将最先进方法的平均点到表面（P2S）精度降低了 50％以上，以 512 体积分辨率实现了约 2mm 的准确度。此外，我们渲染的纹理模型与最先进方法相比具有更高的峰值信噪比（PSNR）。