Abstract
Weakly-supervised medical image segmentation is a challenging task that aimsto reduce the annotation cost while keep the segmentation performance. In thispaper, we present a novel framework, SimTxtSeg, that leverages simple text cuesto generate high-quality pseudo-labels and study the cross-modal fusion intraining segmentation models, simultaneously. Our contribution consists of twokey components: an effective Textual-to-Visual Cue Converter that producesvisual prompts from text prompts on medical images, and a text-guidedsegmentation model with Text-Vision Hybrid Attention that fuses text and imagefeatures. We evaluate our framework on two medical image segmentation tasks:colonic polyp segmentation and MRI brain tumor segmentation, and achieveconsistent state-of-the-art performance.