Abstract
We present Uni$^2$Det, a brand new framework for unified and universalmulti-dataset training on 3D detection, enabling robust performance acrossdiverse domains and generalization to unseen domains. Due to substantialdisparities in data distribution and variations in taxonomy across diversedomains, training such a detector by simply merging datasets poses asignificant challenge. Motivated by this observation, we introduce multi-stageprompting modules for multi-dataset 3D detection, which leverages prompts basedon the characteristics of corresponding datasets to mitigate existingdifferences. This elegant design facilitates seamless plug-and-play integrationwithin various advanced 3D detection frameworks in a unified manner, while alsoallowing straightforward adaptation for universal applicability acrossdatasets. Experiments are conducted across multiple dataset consolidationscenarios involving KITTI, Waymo, and nuScenes, demonstrating that ourUni$^2$Det outperforms existing methods by a large margin in multi-datasettraining. Notably, results on zero-shot cross-dataset transfer validate thegeneralization capability of our proposed method.