Abstract
How do we build a general and broad object detection system? We use alllabels of all concepts ever annotated. These labels span diverse datasets withpotentially inconsistent taxonomies. In this paper, we present a simple methodfor training a unified detector on multiple large-scale datasets. We usedataset-specific training protocols and losses, but share a common detectionarchitecture with dataset-specific outputs. We show how to automaticallyintegrate these dataset-specific outputs into a common semantic taxonomy. Incontrast to prior work, our approach does not require manual taxonomyreconciliation. Our multi-dataset detector performs as well as dataset-specificmodels on each training domain, but generalizes much better to new unseendomains. Entries based on the presented methodology ranked first in the objectdetection and instance segmentation tracks of the ECCV 2020 Robust VisionChallenge.