3D human generation is increasingly significant in various applications.
However, the direct use of 2D generative methods in 3D generation often results
in significant loss of local details, while methods that reconstruct geometry
from generated images struggle with global view consistency. In this work, we
introduce Joint2Human, a novel method that leverages 2D diffusion models to
generate detailed 3D human geometry directly, ensuring both global structure
and local details. To achieve this, we employ the Fourier occupancy field (FOF)
representation, enabling the direct production of 3D shapes as preliminary
results using 2D generative models. With the proposed high-frequency enhancer
and the multi-view recarving strategy, our method can seamlessly integrate the
details from different views into a uniform global shape.To better utilize the
3D human prior and enhance control over the generated geometry, we introduce a
compact spherical embedding of 3D joints. This allows for effective application
of pose guidance during the generation process. Additionally, our method is
capable of generating 3D humans guided by textual inputs. Our experimental
results demonstrate the capability of our method to ensure global structure,
local details, high resolution, and low computational cost, simultaneously.
More results and code can be found on our project page at
this http URL

通过 2D 扩散模型直接生成详细的 3D 人体几何结构，利用 Fourier 占据场表示法产生 3D 形状，并使用紧凑的球面嵌入 3D 关节，实现更好的控制和全局结构的生成，同时保持局部细节、高分辨率和低计算成本。

通过紧凑球面嵌入三维关节实现高质量三维人体生成

Joint2Human: High-quality 3D Human Generation via Compact Spherical  Embedding of 3D Joints

Reconstructing 3D human appearance from a single image is crucial for
achieving holographic communication and immersive social experiences. However,
this remains a challenge for existing methods, which typically rely on
multi-camera setups or are limited to offline operations. In this paper, we
propose R$^2$Human, the first approach for real-time inference and rendering of
photorealistic 3D human appearance from a single image. The core of our
approach is to combine the strengths of implicit texture fields and explicit
neural rendering with our novel representation, namely Z-map. Based on this, we
present an end-to-end network that performs high-fidelity color reconstruction
of visible areas and provides reliable color inference for occluded regions. To
further enhance the 3D perception ability of our network, we leverage the
Fourier occupancy field to reconstruct a detailed 3D geometry, which serves as
a prior for the texture field generation and provides a sampling surface in the
rendering stage. Experiments show that our end-to-end method achieves
state-of-the-art performance on both synthetic data and challenging real-world
images and even outperforms many offline methods. The project page is available
for research purposes at this http URL

使用单幅图像进行实时推断和渲染逼真的三维人体外貌的 R$^2$Human 方法结合了隐式纹理场和显式神经渲染的优势，利用新的表示法 Z-map 并通过傅里叶占用场重建了精细的三维几何，从而在合成数据和挑战性的现实世界图像上取得了最先进的性能。

R2Human：来自单一图像的实时 3D 人类外貌渲染

R2Human: Real-Time 3D Human Appearance Rendering from a Single Image

The advent of deep learning has led to significant progress in monocular
human reconstruction. However, existing representations, such as parametric
models, voxel grids, meshes and implicit neural representations, have
difficulties achieving high-quality results and real-time speed at the same
time. In this paper, we propose Fourier Occupancy Field (FOF), a novel
powerful, efficient and flexible 3D representation, for monocular real-time and
accurate human reconstruction. The FOF represents a 3D object with a 2D field
orthogonal to the view direction where at each 2D position the occupancy field
of the object along the view direction is compactly represented with the first
few terms of Fourier series, which retains the topology and neighborhood
relation in the 2D domain. A FOF can be stored as a multi-channel image, which
is compatible with 2D convolutional neural networks and can bridge the gap
between 3D geometries and 2D images. The FOF is very flexible and extensible,
e.g., parametric models can be easily integrated into a FOF as a prior to
generate more robust results. Based on FOF, we design the first 30+FPS
high-fidelity real-time monocular human reconstruction framework. We
demonstrate the potential of FOF on both public dataset and real captured data.
The code will be released for research purposes.

本文提出了一种新型、强大、高效、灵活的三维表示 —— 傅里叶占用场（FOF），用于单目实时准确的人体重建，并且构建了第一个 30+FPS 高保真实时单目人体重建框架，能够将三维几何体和二维图像之间的鸿沟连结起来。