The potential of digital twin technology, involving the creation of precise
digital replicas of physical objects, to reshape AR experiences in 3D object
tracking and localization scenarios is significant. However, enabling 3D object
tracking with subcentimeter accuracy in dynamic mobile AR environments remains
a formidable challenge. These scenarios often require a more robust pose
estimator capable of handling the inherent sensor-level measurement noise. In
this paper, recognizing the absence of comprehensive solutions in existing
literature, we build upon our previous work, the Digital Twin Tracking Dataset
(DTTD), to address these challenges in mobile AR settings. Specifically, we
propose a transformer-based 6DoF pose estimator designed to withstand the
challenges posed by noisy depth data. Simultaneously, we introduce a novel RGBD
dataset captured using a cutting-edge mobile sensor, the iPhone 14 Pro,
expanding the applicability of our approach to iPhone sensor data. Through
extensive experimentation and in-depth analysis, we illustrate the
effectiveness of our methods in the face of significant depth data errors,
surpassing the performance of existing baselines. Code will be made publicly
available.

数字孪生技术在移动增强现实环境中，通过创建精确的物理对象数字副本，对于重塑三维物体追踪和定位场景具有重要潜力。本文基于我们之前的工作《数字孪生追踪数据集》（DTTD），针对移动增强现实环境中的挑战，提出了一种基于转换器的 6 自由度姿态估计器，并引入了使用 iPhone 14 Pro 的创新 RGBD 数据集，通过大量实验和深入分析，证明了我们方法在处理深度数据错误时的有效性，超过现有基线的性能。代码将公开提供。

基于 RGBD 变形模型和综合移动数据集的亚厘米精度数字双生跟踪

Towards Subcentimeter Accuracy Digital-Twin Tracking via An RGBD-based  Transformer Model and A Comprehensive Mobile Dataset

High-quality 3D ground-truth shapes are critical for 3D object reconstruction
evaluation. However, it is difficult to create a replica of an object in
reality, and even 3D reconstructions generated by 3D scanners have artefacts
that cause biases in evaluation. To address this issue, we introduce a novel
multi-view RGBD dataset captured using a mobile device, which includes highly
precise 3D ground-truth annotations for 153 object models featuring a diverse
set of 3D structures. We obtain precise 3D ground-truth shape without relying
on high-end 3D scanners by utilising LEGO models with known geometry as the 3D
structures for image capture. The distinct data modality offered by
high-resolution RGB images and low-resolution depth maps captured on a mobile
device, when combined with precise 3D geometry annotations, presents a unique
opportunity for future research on high-fidelity 3D reconstruction.
Furthermore, we evaluate a range of 3D reconstruction algorithms on the
proposed dataset. Project page: this http URL

通过利用具备已知几何形状的乐高模型，在手机采集的高分辨率 RGB 图像和低分辨率深度图的基础上，得到了 153 个 3D 对象模型的高精度 3D 真实形状，该多视角 RGBD 数据集可在 3D 重建领域提供独特机会，对多种 3D 重建算法进行了评估。