Abstract
Image-text models excel at image-level tasks but struggle with detailedvisual understanding. While these models provide strong visual-languagealignment, segmentation models like SAM2 offer precise spatial boundaries forobjects. To this end, we propose TextRegion, a simple, effective, andtraining-free framework that combines the strengths of image-text models andSAM2 to generate powerful text-aligned region tokens. These tokens enabledetailed visual understanding while preserving open-vocabulary capabilities.They can be directly applied to various downstream tasks, including open-worldsemantic segmentation, referring expression comprehension, and grounding. Weconduct extensive evaluations and consistently achieve superior or competitiveperformance compared to state-of-the-art training-free methods. Additionally,our framework is compatible with many image-text models, making it highlypractical and easily extensible as stronger models emerge. Code is availableat: https://github.com/avaxiao/TextRegion.