Abstract
We present DexCanvas, a large-scale hybrid real-synthetic human manipulationdataset containing 7,000 hours of dexterous hand-object interactions seededfrom 70 hours of real human demonstrations, organized across 21 fundamentalmanipulation types based on the Cutkosky taxonomy. Each entry combinessynchronized multi-view RGB-D, high-precision mocap with MANO hand parameters,and per-frame contact points with physically consistent force profiles. Ourreal-to-sim pipeline uses reinforcement learning to train policies that controlan actuated MANO hand in physics simulation, reproducing human demonstrationswhile discovering the underlying contact forces that generate the observedobject motion. DexCanvas is the first manipulation dataset to combinelarge-scale real demonstrations, systematic skill coverage based on establishedtaxonomies, and physics-validated contact annotations. The dataset canfacilitate research in robotic manipulation learning, contact-rich control, andskill transfer across different hand morphologies.