Image-based volumetric humans using pixel-aligned features promise
generalization to unseen poses and identities. Prior work leverages global
spatial encodings and multi-view geometric consistency to reduce spatial
ambiguity. However, global encodings often suffer from overfitting to the
distribution of the training data, and it is difficult to learn multi-view
consistent reconstruction from sparse views. In this work, we investigate
common issues with existing spatial encodings and propose a simple yet highly
effective approach to modeling high-fidelity volumetric humans from sparse
views. One of the key ideas is to encode relative spatial 3D information via
sparse 3D keypoints. This approach is robust to the sparsity of viewpoints and
cross-dataset domain gap. Our approach outperforms state-of-the-art methods for
head reconstruction. On human body reconstruction for unseen subjects, we also
achieve performance comparable to prior work that uses a parametric human body
model and temporal feature aggregation. Our experiments show that a majority of
errors in prior work stem from an inappropriate choice of spatial encoding and
thus we suggest a new direction for high-fidelity image-based human modeling.
this https URL

本文提出了一种新的人体建模方法，使用稀疏 3D 关键点编码相对空间 3D 信息，并在头部重建方面超过先前的最先进方法。同时也在人体重建方面性能可与参数化模型与时间特征聚合的先前工作相比拟，实验结果表明提出方法的高保真度建模指向了一个新的方向。