Abstract
We present Boundary IoU (Intersection-over-Union), a new segmentationevaluation measure focused on boundary quality. We perform an extensiveanalysis across different error types and object sizes and show that BoundaryIoU is significantly more sensitive than the standard Mask IoU measure toboundary errors for large objects and does not over-penalize errors on smallerobjects. The new quality measure displays several desirable characteristicslike symmetry w.r.t. prediction/ground truth pairs and balanced responsivenessacross scales, which makes it more suitable for segmentation evaluation thanother boundary-focused measures like Trimap IoU and F-measure. Based onBoundary IoU, we update the standard evaluation protocols for instance andpanoptic segmentation tasks by proposing the Boundary AP (Average Precision)and Boundary PQ (Panoptic Quality) metrics, respectively. Our experiments showthat the new evaluation metrics track boundary quality improvements that aregenerally overlooked by current Mask IoU-based evaluation metrics. We hope thatthe adoption of the new boundary-sensitive evaluation metrics will lead torapid progress in segmentation methods that improve boundary quality.