Augmented reality for laparoscopic liver resection is a visualisation mode
that allows a surgeon to localise tumours and vessels embedded within the liver
by projecting them on top of a laparoscopic image. Preoperative 3D models
extracted from CT or MRI data are registered to the intraoperative laparoscopic
images during this process. In terms of 3D-2D fusion, most of the algorithms
make use of anatomical landmarks to guide registration. These landmarks include
the liver's inferior ridge, the falciform ligament, and the occluding contours.
They are usually marked by hand in both the laparoscopic image and the 3D
model, which is time-consuming and may contain errors if done by a
non-experienced user. Therefore, there is a need to automate this process so
that augmented reality can be used effectively in the operating room. We
present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge
(P2ILF), held during the Medical Imaging and Computer Assisted Interventions
(MICCAI 2022) conference, which investigates the possibilities of detecting
these landmarks automatically and using them in registration. The challenge was
divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D
registration task. The teams were provided with training data consisting of 167
laparoscopic images and 9 preoperative 3D models from 9 patients, with the
corresponding 2D and 3D landmark annotations. A total of 6 teams from 4
countries participated, whose proposed methods were evaluated on 16 images and
two preoperative 3D models from two patients. All the teams proposed deep
learning-based methods for the 2D and 3D landmark segmentation tasks and
differentiable rendering-based methods for the registration task. Based on the
experimental outcomes, we propose three key hypotheses that determine current
limitations and future directions for research in this domain.

在腔镜肝切除手术中，增强现实是一种可视化模式，能够帮助医生在腔镜图像上投影肿瘤和嵌入在肝脏内的血管，通过将其映射到术中腔镜图像中，从而实现定位。该研究主要针对 2D 和 3D 标志物自动检测和标记，以及 3D-2D 配准任务进行深入探讨，并提出了当前限制和未来研究方向。