Abstract
Accurate 3D trajectory data is crucial for advancing autonomous driving. Yet,traditional datasets are usually captured by fixed sensors mounted on a car andare susceptible to occlusion. Additionally, such an approach can preciselyreconstruct the dynamic environment in the close vicinity of the measurementvehicle only, while neglecting objects that are further away. In this paper, weintroduce the DeepScenario Open 3D Dataset (DSC3D), a high-quality,occlusion-free dataset of 6 degrees of freedom bounding box trajectoriesacquired through a novel monocular camera drone tracking pipeline. Our datasetincludes more than 175,000 trajectories of 14 types of traffic participants andsignificantly exceeds existing datasets in terms of diversity and scale,containing many unprecedented scenarios such as complex vehicle-pedestrianinteraction on highly populated urban streets and comprehensive parkingmaneuvers from entry to exit. DSC3D dataset was captured in five variouslocations in Europe and the United States and include: a parking lot, a crowdedinner-city, a steep urban intersection, a federal highway, and a suburbanintersection. Our 3D trajectory dataset aims to enhance autonomous drivingsystems by providing detailed environmental 3D representations, which couldlead to improved obstacle interactions and safety. We demonstrate its utilityacross multiple applications including motion prediction, motion planning,scenario mining, and generative reactive traffic agents. Our interactive onlinevisualization platform and the complete dataset are publicly available athttps://app.deepscenario.com, facilitating research in motion prediction,behavior modeling, and safety validation.