Abstract
While the Segment Anything Model (SAM) excels in semantic segmentation forgeneral-purpose images, its performance significantly deteriorates when appliedto medical images, primarily attributable to insufficient representation ofmedical images in its training dataset. Nonetheless, gathering comprehensivedatasets and training models that are universally applicable is particularlychallenging due to the long-tail problem common in medical images. To addressthis gap, here we present a Self-Sampling Meta SAM (SSM-SAM) framework forfew-shot medical image segmentation. Our innovation lies in the design of threekey modules: 1) An online fast gradient descent optimizer, further optimized bya meta-learner, which ensures swift and robust adaptation to new tasks. 2) ASelf-Sampling module designed to provide well-aligned visual prompts forimproved attention allocation; and 3) A robust attention-based decoderspecifically designed for medical few-shot learning to capture relationshipbetween different slices. Extensive experiments on a popular abdominal CTdataset and an MRI dataset demonstrate that the proposed method achievessignificant improvements over state-of-the-art methods in few-shotsegmentation, with an average improvements of 10.21% and 1.80% in terms of DSC,respectively. In conclusion, we present a novel approach for rapid onlineadaptation in interactive image segmentation, adapting to a new organ in just0.83 minutes. Code is publicly available on GitHub upon acceptance.