SimTxtSeg: Weakly-Supervised Medical Image Segmentation with Simple Text Cues

  • 2024-06-28 06:56:08
  • Yuxin Xie, Tao Zhou, Yi Zhou, Geng Chen
  • 0


Weakly-supervised medical image segmentation is a challenging task that aimsto reduce the annotation cost while keep the segmentation performance. In thispaper, we present a novel framework, SimTxtSeg, that leverages simple text cuesto generate high-quality pseudo-labels and study the cross-modal fusion intraining segmentation models, simultaneously. Our contribution consists of twokey components: an effective Textual-to-Visual Cue Converter that producesvisual prompts from text prompts on medical images, and a text-guidedsegmentation model with Text-Vision Hybrid Attention that fuses text and imagefeatures. We evaluate our framework on two medical image segmentation tasks:colonic polyp segmentation and MRI brain tumor segmentation, and achieveconsistent state-of-the-art performance.


Quick Read (beta)

loading the full paper ...