This paper proposes an end-to-end framework for generating 3D human pose
datasets using Neural Radiance Fields (NeRF). Public datasets generally have
limited diversity in terms of human poses and camera viewpoints, largely due to
the resource-intensive nature of collecting 3D human pose data. As a result,
pose estimators trained on public datasets significantly underperform when
applied to unseen out-of-distribution samples. Previous works proposed
augmenting public datasets by generating 2D-3D pose pairs or rendering a large
amount of random data. Such approaches either overlook image rendering or
result in suboptimal datasets for pre-trained models. Here we propose PoseGen,
which learns to generate a dataset (human 3D poses and images) with a feedback
loss from a given pre-trained pose estimator. In contrast to prior art, our
generated data is optimized to improve the robustness of the pre-trained model.
The objective of PoseGen is to learn a distribution of data that maximizes the
prediction error of a given pre-trained model. As the learned data distribution
contains OOD samples of the pre-trained model, sampling data from such a
distribution for further fine-tuning a pre-trained model improves the
generalizability of the model. This is the first work that proposes NeRFs for
3D human data generation. NeRFs are data-driven and do not require 3D scans of
humans. Therefore, using NeRF for data generation is a new direction for
convenient user-specific data generation. Our extensive experiments show that
the proposed PoseGen improves two baseline models (SPIN and HybrIK) on four
datasets with an average 6% relative improvement.

该研究提出了一种使用神经辐射场（NeRF）生成 3D 人体姿势数据集的端到端框架，该方法通过训练预训练的姿势估计器优化生成的数据分布，从而提高模型的鲁棒性。实验证明，该方法相对于基准模型提高了 6% 的预测准确率。