Abstract
The Segment Anything Model (SAM) represents a state-of-the-art researchadvancement in natural image segmentation, achieving impressive results withinput prompts such as points and bounding boxes. However, our evaluation andrecent research indicate that directly applying the pretrained SAM to medicalimage segmentation does not yield satisfactory performance. This limitationprimarily arises from significant domain gap between natural images and medicalimages. To bridge this gap, we introduce SAM-Med2D, the most comprehensivestudies on applying SAM to medical 2D images. Specifically, we first collectand curate approximately 4.6M images and 19.7M masks from public and privatedatasets, constructing a large-scale medical image segmentation datasetencompassing various modalities and objects. Then, we comprehensively fine-tuneSAM on this dataset and turn it into SAM-Med2D. Unlike previous methods thatonly adopt bounding box or point prompts as interactive segmentation approach,we adapt SAM to medical image segmentation through more comprehensive promptsinvolving bounding boxes, points, and masks. We additionally fine-tune theencoder and decoder of the original SAM to obtain a well-performed SAM-Med2D,leading to the most comprehensive fine-tuning strategies to date. Finally, weconducted a comprehensive evaluation and analysis to investigate theperformance of SAM-Med2D in medical image segmentation across variousmodalities, anatomical structures, and organs. Concurrently, we validated thegeneralization capability of SAM-Med2D on 9 datasets from MICCAI 2023challenge. Overall, our approach demonstrated significantly superiorperformance and generalization capability compared to SAM.