Abstract
People often use physical intuition when manipulating articulated objects,irrespective of object semantics. Motivated by this observation, we identify animportant embodied task where an agent must play with objects to recover theirparts. To this end, we introduce Act the Part (AtP) to learn how to interactwith articulated objects to discover and segment their pieces. By couplingaction selection and motion segmentation, AtP is able to isolate structures tomake perceptual part recovery possible without semantic labels. Our experimentsshow AtP learns efficient strategies for part discovery, can generalize tounseen categories, and is capable of conditional reasoning for the task.Although trained in simulation, we show convincing transfer to real world datawith no fine-tuning.