Given the image collection of an object, we aim at building a real-time
image-based pose estimation method, which requires neither its CAD model nor
hours of object-specific training. Recent NeRF-based methods provide a
promising solution by directly optimizing the pose from pixel loss between
rendered and target images. However, during inference, they require long
converging time, and suffer from local minima, making them impractical for
real-time robot applications. We aim at solving this problem by marrying image
matching with NeRF. With 2D matches and depth rendered by NeRF, we directly
solve the pose in one step by building 2D-3D correspondences between target and
initial view, thus allowing for real-time prediction. Moreover, to improve the
accuracy of 2D-3D correspondences, we propose a 3D consistent point mining
strategy, which effectively discards unfaithful points reconstruted by NeRF.
Moreover, current NeRF-based methods naively optimizing pixel loss fail at
occluded images. Thus, we further propose a 2D matches based sampling strategy
to preclude the occluded area. Experimental results on representative datasets
prove that our method outperforms state-of-the-art methods, and improves
inference efficiency by 90x, achieving real-time prediction at 6 FPS.

基于图像的姿态估计方法，结合 NeRF 技术实现了实时的机器人应用，通过建立 2D-3D 对应关系和改进的点挖掘策略提高了准确性，并且在代表性数据集上的实验结果表明，相比于现有方法，推断效率提高了 90 倍，能够以 6 帧每秒的速度进行实时预测。

将 NeRF 与特征匹配相结合的一步姿态估计

Marrying NeRF with Feature Matching for One-step Pose Estimation

State-of-the-art research of traditional computer vision is increasingly
leveraged in the surgical domain. A particular focus in computer-assisted
surgery is to replace marker-based tracking systems for instrument localization
with pure image-based 6DoF pose estimation. However, the state of the art has
not yet met the accuracy required for surgical navigation. In this context, we
propose a high-fidelity marker-less optical tracking system for surgical
instrument localization. We developed a multi-view camera setup consisting of
static and mobile cameras and collected a large-scale RGB-D video dataset with
dedicated synchronization and data fusions methods. Different state-of-the-art
pose estimation methods were integrated into a deep learning pipeline and
evaluated on multiple camera configurations. Furthermore, the performance
impacts of different input modalities and camera positions, as well as training
on purely synthetic data, were compared. The best model achieved an average
position and orientation error of 1.3 mm and 1.0{\deg} for a surgical drill as
well as 3.8 mm and 5.2{\deg} for a screwdriver. These results significantly
outperform related methods in the literature and are close to clinical-grade
accuracy, demonstrating that marker-less tracking of surgical instruments is
becoming a feasible alternative to existing marker-based systems.

提出了一种高保真度的非标记光学跟踪系统，用于手术器械的本地化，采用多视角相机布置，利用深度学习方法和数据融合技术，对不同输入模型和相机位置进行评估和比较，并且取得了比文献中相关方法更好的平均误差结果，从而展示了手术器械无标记跟踪已经成为了现实。