Weakly-supervised medical image segmentation is a challenging task that aims
to reduce the annotation cost while keep the segmentation performance. In this
paper, we present a novel framework, SimTxtSeg, that leverages simple text cues
to generate high-quality pseudo-labels and study the cross-modal fusion in
training segmentation models, simultaneously. Our contribution consists of two
key components: an effective Textual-to-Visual Cue Converter that produces
visual prompts from text prompts on medical images, and a text-guided
segmentation model with Text-Vision Hybrid Attention that fuses text and image
features. We evaluate our framework on two medical image segmentation tasks:
colonic polyp segmentation and MRI brain tumor segmentation, and achieve
consistent state-of-the-art performance.

本研究提出了一种利用简单文本提示生成高质量伪标签、并研究训练分割模型中的跨模态融合的新框架 SimTxtSeg，其中包括两个关键组件：从医学图像的文本提示中产生视觉提示的有效的文本 - 视觉提示转换器，以及融合文本和图像特征的文本引导分割模型，通过在结肠息肉分割和 MRI 脑肿瘤分割的两个医学图像分割任务上进行评估，达到了始终如一的最新性能。