This paper describes recent developments in object specific pose and shape
prediction from single images. The main contribution is a new approach to
camera pose prediction by self-supervised learning of keypoints corresponding
to locations on a category specific deformable shape. We designed a network to
generate a proxy ground-truth heatmap from a set of keypoints distributed all
over the category-specific mean shape, where each is represented by a unique
color on a labeled texture. The proxy ground-truth heatmap is used to train a
deep keypoint prediction network, which can be used in online inference. The
proposed approach to camera pose prediction show significant improvements when
compared with state-of-the-art methods.
Our approach to camera pose prediction is used to infer 3D objects from 2D
image frames of video sequences online. To train the reconstruction model, it
receives only a silhouette mask from a single frame of a video sequence in
every training step and a category-specific mean object shape. We conducted
experiments using three different datasets representing the bird category: the
CUB [51] image dataset, YouTubeVos and the Davis video datasets. The network is
trained on the CUB dataset and tested on all three datasets. The online
experiments are demonstrated on YouTubeVos and Davis [56] video sequences using
a network trained on the CUB training set.

本文提出了一种基于自监督学习的关键点预测方法来进行相机位姿预测及三维目标物体的在线重建的深度学习网络，并在几个数据集上进行了实验，取得了显著的改进。