In 3D human shape and pose estimation from a monocular video, models trained
with limited labeled data cannot generalize well to videos with occlusion,
which is common in the wild videos. The recent human neural rendering
approaches focusing on novel view synthesis initialized by the off-the-shelf
human shape and pose methods have the potential to correct the initial human
shape. However, the existing methods have some drawbacks such as, erroneous in
handling occlusion, sensitive to inaccurate human segmentation, and ineffective
loss computation due to the non-regularized opacity field. To address these
problems, we introduce ORTexME, an occlusion-robust temporal method that
utilizes temporal information from the input video to better regularize the
occluded body parts. While our ORTexME is based on NeRF, to determine the
reliable regions for the NeRF ray sampling, we utilize our novel average
texture learning approach to learn the average appearance of a person, and to
infer a mask based on the average texture. In addition, to guide the
opacity-field updates in NeRF to suppress blur and noise, we propose the use of
human body mesh. The quantitative evaluation demonstrates that our method
achieves significant improvement on the challenging multi-person 3DPW dataset,
where our method achieves 1.8 P-MPJPE error reduction. The SOTA rendering-based
methods fail and enlarge the error up to 5.6 on the same dataset.

通过利用输入视频的时间信息更好地规范遮挡的身体部位，我们引入 ORTexME，一种具有遮挡鲁棒性的时间方法。我们的方法在具有挑战性的多人 3DPW 数据集上取得了显著改进，P-MPJPE 误差减少 1.8，而渲染方法则使误差增大 5.6。