We present EgoTAP, a heatmap-to-3D pose lifting method for highly accurate
stereo egocentric 3D pose estimation. Severe self-occlusion and out-of-view
limbs in egocentric camera views make accurate pose estimation a challenging
problem. To address the challenge, prior methods employ joint
heatmaps-probabilistic 2D representations of the body pose, but heatmap-to-3D
pose conversion still remains an inaccurate process. We propose a novel
heatmap-to-3D lifting method composed of the Grid ViT Encoder and the
Propagation Network. The Grid ViT Encoder summarizes joint heatmaps into
effective feature embedding using self-attention. Then, the Propagation Network
estimates the 3D pose by utilizing skeletal information to better estimate the
position of obscure joints. Our method significantly outperforms the previous
state-of-the-art qualitatively and quantitatively demonstrated by a 23.9\%
reduction of error in an MPJPE metric. Our source code is available in GitHub.

提出了 EgoTAP 方法，用于高精度的立体自我中心 3D 姿势估计，通过使用自我注意力将联合热力图总结为有效的特征嵌入，然后利用骨骼信息来更好地估计模糊关节的位置，明显优于先前的最先进方法，定量和定性地证明了 MPJPE 度量误差减少 23.9％。

自注意传播网络用于自我中心热图到 3D 姿势提升

Attention-Propagation Network for Egocentric Heatmap to 3D Pose Lifting

Video frame interpolation (VFI) works generally predict intermediate frame(s)
by first estimating the motion between inputs and then warping the inputs to
the target time with the estimated motion. This approach, however, is not
optimal when the temporal distance between the input sequence increases as
existing motion estimation modules cannot effectively handle large motions.
Hence, VFI works perform well for small frame gaps and perform poorly as the
frame gap increases. In this work, we propose a novel framework to address this
problem. We argue that when there is a large gap between inputs, instead of
estimating imprecise motion that will eventually lead to inaccurate
interpolation, we can safely propagate from one side of the input up to a
reliable time frame using the other input as a reference. Then, the rest of the
intermediate frames can be interpolated using standard approaches as the
temporal gap is now narrowed. To this end, we propose a propagation network
(PNet) by extending the classic feature-level forecasting with a novel
motion-to-feature approach. To be thorough, we adopt a simple interpolation
model along with PNet as our full model and design a simple procedure to train
the full model in an end-to-end manner. Experimental results on several
benchmark datasets confirm the effectiveness of our method for long-term VFI
compared to state-of-the-art approaches.

本研究提出了一种基于传播网络（PNet）的新框架，旨在有效解决视频帧插值（VFI）中存在的大间隔问题，将一侧的输入传播到可靠时间帧，然后使用标准方法进行插值。此方法在多项基准测试数据集上的实验结果表明，在长期 VFI 方面，其比现有的最先进方法更加有效。