Learning deformable 3D objects from 2D images is often an ill-posed problem.
Existing methods rely on explicit supervision to establish multi-view
correspondences, such as template shape models and keypoint annotations, which
restricts their applicability on objects "in the wild". A more natural way of
establishing correspondences is by watching videos of objects moving around. In
this paper, we present DOVE, a method that learns textured 3D models of
deformable object categories from monocular videos available online, without
keypoint, viewpoint or template shape supervision. By resolving
symmetry-induced pose ambiguities and leveraging temporal correspondences in
videos, the model automatically learns to factor out 3D shape, articulated pose
and texture from each individual RGB frame, and is ready for single-image
inference at test time. In the experiments, we show that existing methods fail
to learn sensible 3D shapes without additional keypoint or template
supervision, whereas our method produces temporally consistent 3D models, which
can be animated and rendered from arbitrary viewpoints.

DOVE 是一种用于从单目视频中学习可变形物体纹理 3D 模型的方法，无需关键点、视点或模板形状监督，并利用视频中的时间对应关系和对称性消除姿态歧义，将三维形状、艺术姿势和纹理从每个单独的 RGB 帧中分离出来。

通过观看视频学习可变形三维物体的 DOVE 模型

DOVE: Learning Deformable 3D Objects by Watching Videos

In this paper, we address the problem of detecting unseen objects from RGB
images and estimating their poses in 3D. We propose two mobile friendly
networks: MobilePose-Base and MobilePose-Shape. The former is used when there
is only pose supervision, and the latter is for the case when shape supervision
is available, even a weak one. We revisit shape features used in previous
methods, including segmentation and coordinate map. We explain when and why
pixel-level shape supervision can improve pose estimation. Consequently, we add
shape prediction as an intermediate layer in the MobilePose-Shape, and let the
network learn pose from shape. Our models are trained on mixed real and
synthetic data, with weak and noisy shape supervision. They are ultra
lightweight that can run in real-time on modern mobile devices (e.g. 36 FPS on
Galaxy S20). Comparing with previous single-shot solutions, our method has
higher accuracy, while using a significantly smaller model (2~3% in model size
or number of parameters).

本文介绍了一种可以实时运行在手机设备上的目标检测和位姿估计方法，该方法通过无监督的形状信息和弱监督的数据训练出 MobilePose-Base 和 MobilePose-Shape 两个超轻量级的网络，相较于之前的单次射击方案具有更高的精度，同时使用更小的模型。