Manipulation of tissue with surgical tools often results in large deformations that current methods in tracking and reconstructing algorithms have not effectively addressed. A major source of tracking errors during large deformations stems from wrong data association between observed sensor measurements with previously tracked scene. To mitigate this issue, we present a surgical perception framework, SuPerPM, that leverages learning-based non-rigid point cloud matching for data association, thus accommodating larger deformations. The learning models typically require training data with ground truth point cloud correspondences, which is challenging or even impractical to collect in surgical environments. Thus, for tuning the learning model, we gather endoscopic data of soft tissue being manipulated by a surgical robot and then establish correspondences between point clouds at different time points to serve as ground truth. This was achieved by employing a position-based dynamics (PBD) simulation to ensure that the correspondences adhered to physical constraints. The proposed framework is demonstrated on several challenging surgical datasets that are characterized by large deformations, achieving superior performance over state-of-the-art surgical scene tracking algorithms.

通过非刚性点云匹配的学习，解决了手术工具操作引起的大变形时跟踪误差的问题，提出了一种手术感知框架（SuPerPM），该框架利用学习模型进行数据关联，适应更大的变形，并在多个具有大变形特征的手术数据集上展示了优越的性能，超越了现有的手术场景跟踪算法。

SuPerPM: 基于深度点匹配的大变形稳健外科感知框架，从物理约束仿真数据学习