This work proposes an end-to-end neural interactive keypoint detection
framework named Click-Pose, which can significantly reduce more than 10 times
labeling costs of 2D keypoint annotation compared with manual-only annotation.
Click-Pose explores how user feedback can cooperate with a neural keypoint
detector to correct the predicted keypoints in an interactive way for a faster
and more effective annotation process. Specifically, we design the pose error
modeling strategy that inputs the ground truth pose combined with four typical
pose errors into the decoder and trains the model to reconstruct the correct
poses, which enhances the self-correction ability of the model. Then, we attach
an interactive human-feedback loop that allows receiving users' clicks to
correct one or several predicted keypoints and iteratively utilizes the decoder
to update all other keypoints with a minimum number of clicks (NoC) for
efficient annotation. We validate Click-Pose in in-domain, out-of-domain
scenes, and a new task of keypoint adaptation. For annotation, Click-Pose only
needs 1.97 and 6.45 NoC@95 (at precision 95%) on COCO and Human-Art, reducing
31.4% and 36.3% efforts than the SOTA model (ViTPose) with manual correction,
respectively. Besides, without user clicks, Click-Pose surpasses the previous
end-to-end model by 1.4 AP on COCO and 3.0 AP on Human-Art. The code is
available at this https URL

该研究提出了一种名为 Click-Pose 的端到端神经交互式关键点检测框架，相比于仅使用手动标注，能够大大降低 2D 关键点注释的标注成本，并探索用户反馈如何与神经关键点检测器合作，在交互方式下更快、更有效地修正预测的关键点。