We present a near real-time method for 6-DoF tracking of an unknown object
from a monocular RGBD video sequence, while simultaneously performing neural 3D
reconstruction of the object. Our method works for arbitrary rigid objects,
even when visual texture is largely absent. The object is assumed to be
segmented in the first frame only. No additional information is required, and
no assumption is made about the interaction agent. Key to our method is a
Neural Object Field that is learned concurrently with a pose graph optimization
process in order to robustly accumulate information into a consistent 3D
representation capturing both geometry and appearance. A dynamic pool of posed
memory frames is automatically maintained to facilitate communication between
these threads. Our approach handles challenging sequences with large pose
changes, partial and full occlusion, untextured surfaces, and specular
highlights. We show results on HO3D, YCBInEOAT, and BEHAVE datasets,
demonstrating that our method significantly outperforms existing approaches.
Project page: this https URL

本研究提出了一种近实时的方法，可以从单目 RGBD 视频序列中对未知物体进行 6 自由度跟踪，并同时进行神经 3D 重建。该方法适用于任意刚体对象，即使视觉纹理基本不存在。该方法的关键在于同时学习神经对象场和姿态图优化过程，以稳健地累积信息以捕捉物体的几何和外观，而不需要其他信息和假设。本方法可以处理具有大的位姿变化，部分遮挡，无纹理表面和高光反射等挑战性序列，而且通过在 HO3D，YCBInEOAT 和 BEHAVE 数据集上的测试，证明本研究方法显著优于现有方法。

BundleSDF：未知物体的 6 自由度神经跟踪和 3D 重建

BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects

We present a challenging and realistic novel dataset for evaluating 6-DOF
object tracking algorithms. Existing datasets show serious
limitations---notably, unrealistic synthetic data, or real data with large
fiducial markers---preventing the community from obtaining an accurate picture
of the state-of-the-art. Using a data acquisition pipeline based on a
commercial motion capture system for acquiring accurate ground truth poses of
real objects with respect to a Kinect V2 camera, we build a dataset which
contains a total of 297 calibrated sequences. They are acquired in three
different scenarios to evaluate the performance of trackers: stability,
robustness to occlusion and accuracy during challenging interactions between a
person and the object. We conduct an extensive study of a deep 6-DOF tracking
architecture and determine a set of optimal parameters. We enhance the
architecture and the training methodology to train a 6-DOF tracker that can
robustly generalize to objects never seen during training, and demonstrate
favorable performance compared to previous approaches trained specifically on
the objects to track.

使用商用运动捕捉系统获取真实物体的高度准确地面真实姿态，并通过三个不同的场景评估了具有稳定性和抗干扰能力的 6-DOF 追踪器的性能，通过深度学习构建的跟踪器的性能对比先前的方法具有较大的优势。