In drug discovery, molecule optimization is an important step in order tomodify drug candidates into better ones in terms of desired drug properties.With the recent advance of Artificial Intelligence, this traditionally in vitroprocess has been increasingly facilitated by in silico approaches. We presentan innovative in silico approach to computationally optimizing molecules andformulate the problem as to generate optimized molecular graphs via deepgenerative models. Our generative models follow the key idea of fragment-baseddrug design, and optimize molecules by modifying their small fragments. Ourmodels learn how to identify the to-be-optimized fragments and how to modifysuch fragments by learning from the difference of molecules that have good andbad properties. In optimizing a new molecule, our models apply the learnedsignals to decode optimized fragments at the predicted location of thefragments. We also construct multiple such models into a pipeline such thateach of the models in the pipeline is able to optimize one fragment, and thusthe entire pipeline is able to modify multiple fragments of molecule if needed.We compare our models with other state-of-the-art methods on benchmark datasetsand demonstrate that our methods significantly outperform others with more than80% property improvement under moderate molecular similarity constraints, andmore than 10% property improvement under high molecular similarity constraints.