Modeling 3D Shapes by Reinforcement Learning

  • 2020-07-20 04:40:18
  • Cheng Lin, Tingxiang Fan, Wenping Wang, Matthias Nie├čner
  • 0


We explore how to enable machines to model 3D shapes like human modelersusing deep reinforcement learning (RL). In 3D modeling software like Maya, amodeler usually creates a mesh model in two steps: (1) approximating the shapeusing a set of primitives; (2) editing the meshes of the primitives to createdetailed geometry. Inspired by such artist-based modeling, we propose atwo-step neural framework based on RL to learn 3D modeling policies. By takingactions and collecting rewards in an interactive environment, the agents firstlearn to parse a target shape into primitives and then to edit the geometry. Toeffectively train the modeling agents, we introduce a novel training algorithmthat combines heuristic policy, imitation learning and reinforcement learning.Our experiments show that the agents can learn good policies to produce regularand structure-aware mesh models, which demonstrates the feasibility andeffectiveness of the proposed RL framework.


Quick Read (beta)

loading the full paper ...