Recent advances in implicit function-based approaches have shown promising
results in 3D human reconstruction from a single RGB image. However, these
methods are not sufficient to extend to more general cases, often generating
dragged or disconnected body parts, particularly for animated characters. We
argue that these limitations stem from the use of the existing point-level 3D
shape representation, which lacks holistic 3D context understanding.
Voxel-based reconstruction methods are more suitable for capturing the entire
3D space at once, however, these methods are not practical for high-resolution
reconstructions due to their excessive memory usage. To address these
challenges, we introduce Tri-directional Implicit Function (TIFu), which is a
vector-level representation that increases global 3D consistencies while
significantly reducing memory usage compared to voxel representations. We also
introduce a new algorithm in 3D reconstruction at an arbitrary resolution by
aggregating vectors along three orthogonal axes, resolving inherent problems
with regressing fixed dimension of vectors. Our approach achieves
state-of-the-art performances in both our self-curated character dataset and
the benchmark 3D human dataset. We provide both quantitative and qualitative
analyses to support our findings.

Tri-directional Implicit Function (TIFu) 是一种矢量级表示方法，通过沿三个正交轴聚合矢量解决了回归固定维度矢量的固有问题，从而在任意分辨率的三维重建中取得了最先进的性能。

TIFu：高保真 3D 角色重建的三向隐式函数

TIFu: Tri-directional Implicit Function for High-Fidelity 3D Character  Reconstruction

Recent advances in convolutional neural networks have shown promising results
in 3D shape completion. But due to GPU memory limitations, these methods can
only produce low-resolution outputs. To inpaint 3D models with semantic
plausibility and contextual details, we introduce a hybrid framework that
combines a 3D Encoder-Decoder Generative Adversarial Network (3D-ED-GAN) and a
Long-term Recurrent Convolutional Network (LRCN). The 3D-ED-GAN is a 3D
convolutional neural network trained with a generative adversarial paradigm to
fill missing 3D data in low-resolution. LRCN adopts a recurrent neural network
architecture to minimize GPU memory usage and incorporates an Encoder-Decoder
pair into a Long Short-term Memory Network. By handling the 3D model as a
sequence of 2D slices, LRCN transforms a coarse 3D shape into a more complete
and higher resolution volume. While 3D-ED-GAN captures global contextual
structure of the 3D shape, LRCN localizes the fine-grained details.
Experimental results on both real-world and synthetic data show reconstructions
from corrupted models result in complete and high-resolution 3D objects.

本文介绍了一种结合了 3D 编码器 - 解码器生成对抗网络（3D-ED-GAN）和长期循环卷积网络（LRCN）的混合框架，以 inpaint 具有语义真实性和上下文细节的 3D 模型，实验结果表明，其能够从损坏的模型中重构出完整且高分辨率的 3D 对象。