Abstract
We present LiDAR-EDIT, a novel paradigm for generating synthetic LiDAR datafor autonomous driving. Our framework edits real-world LiDAR scans byintroducing new object layouts while preserving the realism of the backgroundenvironment. Compared to end-to-end frameworks that generate LiDAR point cloudsfrom scratch, LiDAR-EDIT offers users full control over the object layout,including the number, type, and pose of objects, while keeping most of theoriginal real-world background. Our method also provides object labels for thegenerated data. Compared to novel view synthesis techniques, our frameworkallows for the creation of counterfactual scenarios with object layoutssignificantly different from the original real-world scene. LiDAR-EDIT usesspherical voxelization to enforce correct LiDAR projective geometry in thegenerated point clouds by construction. During object removal and insertion,generative models are employed to fill the unseen background and object partsthat were occluded in the original real LiDAR scans. Experimental resultsdemonstrate that our framework produces realistic LiDAR scans with practicalvalue for downstream tasks.