Abstract
Diffusion models have demonstrated their effectiveness across variousgenerative tasks. However, when applied to medical image segmentation, thesemodels encounter several challenges, including significant resource and timerequirements. They also necessitate a multi-step reverse process and multiplesamples to produce reliable predictions. To address these challenges, weintroduce the first latent diffusion segmentation model, named SDSeg, builtupon stable diffusion (SD). SDSeg incorporates a straightforward latentestimation strategy to facilitate a single-step reverse process and utilizeslatent fusion concatenation to remove the necessity for multiple samples.Extensive experiments indicate that SDSeg surpasses existing state-of-the-artmethods on five benchmark datasets featuring diverse imaging modalities.Remarkably, SDSeg is capable of generating stable predictions with a solitaryreverse step and sample, epitomizing the model's stability as implied by itsname. The code is available athttps://github.com/lin-tianyu/Stable-Diffusion-Seg