Abstract
In this paper, we present a holistically end-to-end algorithm for linesegment detection with transformers that is post-processing andheuristics-guided intermediate processing (edge/junction/region detection)free. Our method, named LinE segment TRansformers (LETR), tackles the threemain problems in this domain, namely edge element detection, perceptualgrouping, and holistic inference by three highlights in detection transformers(DETR) including tokenized queries with integrated encoding and decoding,self-attention, and joint queries respectively. The transformers learn toprogressively refine line segments through layers of self-attention mechanismskipping the heuristic design in the previous line segmentation algorithms. Weequip multi-scale encoder/decoder in the transformers to perform fine-grainedline segment detection under a direct end-point distance loss that isparticularly suitable for entities such as line segments that are notconveniently represented by bounding boxes. In the experiments, we showstate-of-the-art results on Wireframe and YorkUrban benchmarks. LETR points toa promising direction for joint end-to-end detection of general entities beyondthe standard object bounding box representation.