Abstract
The ability to perceive the environments in different ways is essential torobotic research. This involves the analysis of both 2D and 3D data sources. Wepresent a large scale urban scene dataset associated with a handy simulatorbased on Unreal Engine 4 and AirSim, which consists of both man-made andreal-world reconstruction scenes in different scales, referred to asUrbanScene3D. Unlike previous works that purely based on 2D information orman-made 3D CAD models, UrbanScene3D contains both compact man-made models anddetailed real-world models reconstructed by aerial images. Each building hasbeen manually extracted from the entire scene model and then has been assignedwith a unique label, forming an instance segmentation map. The provided 3Dground-truth textured models with instance segmentation labels in UrbanScene3Dallow users to obtain all kinds of data they would like to have: instancesegmentation map, depth map in arbitrary resolution, 3D point cloud/mesh inboth visible and invisible places, etc. In addition, with the help of AirSim,users can also simulate the robots (cars/drones)to test a variety of autonomoustasks in the proposed city environment. Please refer to our paper andwebsite(https://vcc.tech/UrbanScene3D/) for further details and applications.