Abstract
Domain-generalized nuclei segmentation refers to the generalizability ofmodels to unseen domains based on knowledge learned from source domains and ischallenged by various image conditions, cell types, and stain strategies.Recently, the Segment Anything Model (SAM) has made great success in universalimage segmentation by interactive prompt modes (e.g., point and box). Despiteits strengths, the original SAM presents limited adaptation to medical images.Moreover, SAM requires providing manual bounding box prompts for each object toproduce satisfactory segmentation masks, so it is laborious in nucleisegmentation scenarios. To address these limitations, we propose adomain-generalizable framework for nuclei image segmentation, abbreviated toNuSegDG. Specifically, we first devise a Heterogeneous Space Adapter(HS-Adapter) to learn multi-dimensional feature representations of differentnuclei domains by injecting a small number of trainable parameters into theimage encoder of SAM. To alleviate the labor-intensive requirement of manualprompts, we introduce a Gaussian-Kernel Prompt Encoder (GKP-Encoder) togenerate density maps driven by a single point, which guides segmentationpredictions by mixing position prompts and semantic prompts. Furthermore, wepresent a Two-Stage Mask Decoder (TSM-Decoder) to effectively convert semanticmasks to instance maps without the manual demand for morphological shaperefinement. Based on our experimental evaluations, the proposed NuSegDGdemonstrates state-of-the-art performance in nuclei instance segmentation,exhibiting superior domain generalization capabilities. The source code isavailable at https://github.com/xq141839/NuSegDG.