Denoising diffusion probabilistic models that were initially proposed for
realistic image generation have recently shown success in various perception
tasks (e.g., object detection and image segmentation) and are increasingly
gaining attention in computer vision. However, extending such models to
multi-frame human pose estimation is non-trivial due to the presence of the
additional temporal dimension in videos. More importantly, learning
representations that focus on keypoint regions is crucial for accurate
localization of human joints. Nevertheless, the adaptation of the
diffusion-based methods remains unclear on how to achieve such objective. In
this paper, we present DiffPose, a novel diffusion architecture that formulates
video-based human pose estimation as a conditional heatmap generation problem.
First, to better leverage temporal information, we propose SpatioTemporal
Representation Learner which aggregates visual evidences across frames and uses
the resulting features in each denoising step as a condition. In addition, we
present a mechanism called Lookup-based MultiScale Feature Interaction that
determines the correlations between local joints and global contexts across
multiple scales. This mechanism generates delicate representations that focus
on keypoint regions. Altogether, by extending diffusion models, we show two
unique characteristics from DiffPose on pose estimation task: (i) the ability
to combine multiple sets of pose estimates to improve prediction accuracy,
particularly for challenging joints, and (ii) the ability to adjust the number
of iterative steps for feature refinement without retraining the model.
DiffPose sets new state-of-the-art results on three benchmarks: PoseTrack2017,
PoseTrack2018, and PoseTrack21.

DiffPose 是一种新颖的扩展扩散模型，将基于视频的人体姿势估计作为条件热图生成问题，并通过多组姿势估计的结合以及迭代步骤的调整来提高预测准确性，成功地在 PoseTrack2017、PoseTrack2018 和 PoseTrack21 三个基准测试上取得了新的最佳结果。