Foundation models such as the recently introduced Segment Anything Model (SAM) have achieved remarkable results in image segmentation tasks. However, these models typically require user interaction through handcrafted prompts such as bounding boxes, which limits their deployment to downstream tasks. Adapting these models to a specific task with fully labeled data also demands expensive prior user interaction to obtain ground-truth annotations. This work proposes to replace conditioning on input prompts with a lightweight module that directly learns a prompt embedding from the image embedding, both of which are subsequently used by the foundation model to output a segmentation mask. Our foundation models with learnable prompts can automatically segment any specific region by 1) modifying the input through a prompt embedding predicted by a simple module, and 2) using weak labels (tight bounding boxes) and few-shot supervision (10 samples). Our approach is validated on MedSAM, a version of SAM fine-tuned for medical images, with results on three medical datasets in MR and ultrasound imaging. Our code is available on https://github.com/Minimel/MedSAMWeakFewShotPromptAutomation.

本研究解决了传统图像分割模型在任务适应性和用户交互需求方面的不足。提出了一种新的方法，利用轻量级模块直接从图像嵌入中学习提示嵌入，能够在仅有少量样本和弱标注的情况下自动进行区域分割。实验结果表明，该方法在医学影像的三个数据集上表现出色，有望推动医学图像分割技术的发展。

通过弱少量监督学习提示自动化MedSAM