Abstract
Can Transformer perform $2\mathrm{D}$ object-level recognition from a puresequence-to-sequence perspective with minimal knowledge about the $2\mathrm{D}$spatial structure? To answer this question, we present You Only Look at OneSequence (YOLOS), a series of object detection models based on the na\"iveVision Transformer with the fewest possible modifications as well as inductivebiases. We find that YOLOS pre-trained on the mid-sized ImageNet-$1k$ datasetonly can already achieve competitive object detection performance on COCO,\textit{e.g.}, YOLOS-Base directly adopted from BERT-Base can achieve $42.0$box AP. We also discuss the impacts as well as limitations of current pre-trainschemes and model scaling strategies for Transformer in vision through objectdetection. Code and model weights are available at\url{https://github.com/hustvl/YOLOS}.