Abstract
This paper studies the problem of fixing malfunctional 3D objects. Whileprevious works focus on building passive perception models to learn thefunctionality from static 3D objects, we argue that functionality is reckonedwith respect to the physical interactions between the object and the user.Given a malfunctional object, humans can perform mental simulations to reasonabout its functionality and figure out how to fix it. Inspired by this, wepropose FixIt, a dataset that contains about 5k poorly-designed 3D physicalobjects paired with choices to fix them. To mimic humans' mental simulationprocess, we present FixNet, a novel framework that seamlessly incorporatesperception and physical dynamics. Specifically, FixNet consists of a perceptionmodule to extract the structured representation from the 3D point cloud, aphysical dynamics prediction module to simulate the results of interactions on3D objects, and a functionality prediction module to evaluate the functionalityand choose the correct fix. Experimental results show that our frameworkoutperforms baseline models by a large margin, and can generalize well toobjects with similar interaction types.