Open-World Entity Segmentation

  • 2021-07-29 17:59:05
  • Lu Qi, Jason Kuen, Yi Wang, Jiuxiang Gu, Hengshuang Zhao, Zhe Lin, Philip Torr, Jiaya Jia
  • 42


We introduce a new image segmentation task, termed Entity Segmentation (ES)with the aim to segment all visual entities in an image without consideringsemantic category labels. It has many practical applications in imagemanipulation/editing where the segmentation mask quality is typically crucialbut category labels are less important. In this setting, allsemantically-meaningful segments are equally treated as categoryless entitiesand there is no thing-stuff distinction. Based on our unified entityrepresentation, we propose a center-based entity segmentation framework withtwo novel modules to improve mask quality. Experimentally, both our new taskand framework demonstrate superior advantages as against existing work. Inparticular, ES enables the following: (1) merging multiple datasets to form alarge training set without the need to resolve label conflicts; (2) any modeltrained on one dataset can generalize exceptionally well to other datasets withunseen domains. Our code is made publicly available at


Quick Read (beta)

loading the full paper ...