Generative AI, in general, and synthetic visual data generation, in specific,
hold much promise for benefiting surgical training by providing photorealism to
simulation environments. Current training methods primarily rely on reading
materials and observing live surgeries, which can be time-consuming and
impractical. In this work, we take a significant step towards improving the
training process. Specifically, we use diffusion models in combination with a
zero-shot video diffusion method to interactively generate realistic
laparoscopic images and videos by specifying a surgical action through text and
guiding the generation with tool positions through segmentation masks. We
demonstrate the performance of our approach using the publicly available Cholec
dataset family and evaluate the fidelity and factual correctness of our
generated images using a surgical action recognition model as well as the
pixel-wise F1-score for the spatial control of tool generation. We achieve an
FID of 38.097 and an F1-score of 0.71.

通过使用扩散模型结合零样本视频扩散方法，通过文本指定手术动作并通过分割掩模指导生成真实的腹腔镜图像和视频，本研究在提高培训过程中迈出了重要一步，评估了生成的图像的保真度和事实正确性，获得了 FID 指标为 38.097 和 F1-score 指标为 0.71。