Accurately estimating the 3D pose and shape is an essential step towards
understanding animal behavior, and can potentially benefit many downstream
applications, such as wildlife conservation. However, research in this area is
held back by the lack of a comprehensive and diverse dataset with high-quality
3D pose and shape annotations. In this paper, we propose Animal3D, the first
comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D
consists of 3379 images collected from 40 mammal species, high-quality
annotations of 26 keypoints, and importantly the pose and shape parameters of
the SMAL model. All annotations were labeled and checked manually in a
multi-stage process to ensure highest quality results. Based on the Animal3D
dataset, we benchmark representative shape and pose estimation models at: (1)
supervised learning from only the Animal3D data, (2) synthetic to real transfer
from synthetically generated images, and (3) fine-tuning human pose and shape
estimation models. Our experimental results demonstrate that predicting the 3D
shape and pose of animals across species remains a very challenging task,
despite significant advances in human pose estimation. Our results further
demonstrate that synthetic pre-training is a viable strategy to boost the model
performance. Overall, Animal3D opens new directions for facilitating future
research in animal 3D pose and shape estimation, and is publicly available.

提出了 Animal3D 数据集，为哺乳动物的 3D 姿态和形状估计提供了全面的数据集，实验证明跨物种的动物 3D 形状和姿态的预测仍然是一项非常具有挑战性的任务，合成预训练是提高模型性能的一种可行策略。