Generative AI, in general, and synthetic visual data generation, in specific,
hold much promise for benefiting surgical training by providing photorealism to
simulation environments. Current training methods primarily rely on reading
materials and observing live surgeries, which can be time-consuming and
impractical. In this work, we take a significant step towards improving the
training process. Specifically, we use diffusion models in combination with a
zero-shot video diffusion method to interactively generate realistic
laparoscopic images and videos by specifying a surgical action through text and
guiding the generation with tool positions through segmentation masks. We
demonstrate the performance of our approach using the publicly available Cholec
dataset family and evaluate the fidelity and factual correctness of our
generated images using a surgical action recognition model as well as the
pixel-wise F1-score for the spatial control of tool generation. We achieve an
FID of 38.097 and an F1-score of 0.71.

通过使用扩散模型结合零样本视频扩散方法，通过文本指定手术动作并通过分割掩模指导生成真实的腹腔镜图像和视频，本研究在提高培训过程中迈出了重要一步，评估了生成的图像的保真度和事实正确性，获得了 FID 指标为 38.097 和 F1-score 指标为 0.71。

扩散模型的交互式生成腹腔镜视频

Interactive Generation of Laparoscopic Videos with Diffusion Models

Given that a conventional laparoscope only provides a two-dimensional (2-D)
view, the detection and diagnosis of medical ailments can be challenging. To
overcome the visual constraints associated with laparoscopy, the use of
laparoscopic images and videos to reconstruct the three-dimensional (3-D)
anatomical structure of the abdomen has proven to be a promising approach.
Neural Radiance Fields (NeRFs) have recently gained attention thanks to their
ability to generate photorealistic images from a 3-D static scene, thus
facilitating a more comprehensive exploration of the abdomen through the
synthesis of new views. This distinguishes NeRFs from alternative methods such
as Simultaneous Localization and Mapping (SLAM) and depth estimation. In this
paper, we present a comprehensive examination of NeRFs in the context of
laparoscopy surgical videos, with the goal of rendering abdominal scenes in
3-D. Although our experimental results are promising, the proposed approach
encounters substantial challenges, which require further exploration in future
research.

提供一个关键的可视，两个维度的视图，突破 0-D 静态场景，为了生成新的观察。

基于 NeRFs 的腹腔三维场景渲染的研究

Towards Abdominal 3-D Scene Rendering from Laparoscopy Surgical Videos  using NeRFs

In the medical domain, the lack of large training data sets and benchmarks is
often a limiting factor for training deep neural networks. In contrast to
expensive manual labeling, computer simulations can generate large and fully
labeled data sets with a minimum of manual effort. However, models that are
trained on simulated data usually do not translate well to real scenarios. To
bridge the domain gap between simulated and real laparoscopic images, we
exploit recent advances in unpaired image-to-image translation. We extent an
image-to-image translation method to generate a diverse multitude of
realistically looking synthetic images based on images from a simple
laparoscopy simulation. By incorporating means to ensure that the image content
is preserved during the translation process, we ensure that the labels given
for the simulated images remain valid for their realistically looking
translations. This way, we are able to generate a large, fully labeled
synthetic data set of laparoscopic images with realistic appearance. We show
that this data set can be used to train models for the task of liver
segmentation of laparoscopic images. We achieve average dice scores of up to
0.89 in some patients without manually labeling a single laparoscopic image and
show that using our synthetic data to pre-train models can greatly improve
their performance. The synthetic data set will be made publicly available,
fully labeled with segmentation maps, depth maps, normal maps, and positions of
tools and camera (this http URL).

通过图像翻译方法，我们生成了外科镜检查图像的大量合成数据集，并展示了这样的数据集可用于训练和提高肝脏分割模型性能的可能性。

使用非配对的图像转换方式生成用于腹腔镜图像处理任务的大型标注数据集

Generating large labeled data sets for laparoscopic image processing  tasks using unpaired image-to-image translation

In this paper, an automatic approach to predict 3D coordinates from stereo
laparoscopic images is presented. The approach maps a vector of pixel
intensities to 3D coordinates through training a six layer deep neural network.
The architectural aspects of the approach is presented and in detail and the
method is evaluated on a publicly available dataset with promising results.

本文介绍了一种自动预测立体腹腔镜图像的三维坐标的方法，通过训练一个六层深度神经网络，将像素强度的向量映射到三维坐标。该方法的架构方面已经详细介绍，并在公开可用的数据集上进行了评估，结果有很大的发展潜力。