This work focuses on the 3D reconstruction of non-rigid objects based on
monocular RGB video sequences. Concretely, we aim at building high-fidelity
models for generic object categories and casually captured scenes. To this end,
we do not assume known root poses of objects, and do not utilize
category-specific templates or dense pose priors. The key idea of our method,
Root Pose Decomposition (RPD), is to maintain a per-frame root pose
transformation, meanwhile building a dense field with local transformations to
rectify the root pose. The optimization of local transformations is performed
by point registration to the canonical space. We also adapt RPD to multi-object
scenarios with object occlusions and individual differences. As a result, RPD
allows non-rigid 3D reconstruction for complicated scenarios containing objects
with large deformations, complex motion patterns, occlusions, and scale
diversities of different individuals. Such a pipeline potentially scales to
diverse sets of objects in the wild. We experimentally show that RPD surpasses
state-of-the-art methods on the challenging DAVIS, OVIS, and AMA datasets.

基于单目 RGB 视频序列的、非刚性物体的三维重建，通过 Root Pose Decomposition（RPD）方法，实现了对具有大变形、复杂运动模式、遮挡和不同个体尺度差异的复杂场景下物体的三维重建。

基于单目视频的通用非刚性三维重建的根姿态分解

Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with  Monocular Videos

This paper investigates how to realize better and more efficient embedding
learning to tackle the semi-supervised video object segmentation under
challenging multi-object scenarios. The state-of-the-art methods learn to
decode features with a single positive object and thus have to match and
segment each target separately under multi-object scenarios, consuming multiple
times computing resources. To solve the problem, we propose an Associating
Objects with Transformers (AOT) approach to match and decode multiple objects
uniformly. In detail, AOT employs an identification mechanism to associate
multiple targets into the same high-dimensional embedding space. Thus, we can
simultaneously process multiple objects' matching and segmentation decoding as
efficiently as processing a single object. For sufficiently modeling
multi-object association, a Long Short-Term Transformer is designed for
constructing hierarchical matching and propagation. We conduct extensive
experiments on both multi-object and single-object benchmarks to examine AOT
variant networks with different complexities. Particularly, our R50-AOT-L
outperforms all the state-of-the-art competitors on three popular benchmarks,
i.e., YouTube-VOS (84.1% J&F), DAVIS 2017 (84.9%), and DAVIS 2016 (91.1%),
while keeping more than $3\times$ faster multi-object run-time. Meanwhile, our
AOT-T can maintain real-time multi-object speed on the above benchmarks. Based
on AOT, we ranked 1st in the 3rd Large-scale VOS Challenge.

本研究提出了基于 AOT 的嵌入式学习方法，通过将多个对象相互关联在同一高维度嵌入空间中，实现了同时处理多个对象的匹配和分割解码，具有更高的效率，并且在多个基准测试中均优于已有的 State-of-the-art 算法，是第三次大规模 VOS 挑战的第一名。