RGBD-based real-time dynamic 3D reconstruction suffers from inaccurate
inter-frame motion estimation as errors may accumulate with online tracking.
This problem is even more severe for single-view-based systems due to strong
occlusions. Based on these observations, we propose OcclusionFusion, a novel
method to calculate occlusion-aware 3D motion to guide the reconstruction. In
our technique, the motion of visible regions is first estimated and combined
with temporal information to infer the motion of the occluded regions through
an LSTM-involved graph neural network. Furthermore, our method computes the
confidence of the estimated motion by modeling the network output with a
probabilistic model, which alleviates untrustworthy motions and enables robust
tracking. Experimental results on public datasets and our own recorded data
show that our technique outperforms existing single-view-based real-time
methods by a large margin. With the reduction of the motion errors, the
proposed technique can handle long and challenging motion sequences. Please
check out the project page for sequence results:
this https URL

本文提出了一种名为 OcclusionFusion 的新方法，使用一种基于 LSTM 图神经网络的遮挡感知 3D 运动来指导实时重建，实验证明该方法的性能比现有的基于单视角的方法要好得多。