Process synthesis experiences a disruptive transformation accelerated bydigitization and artificial intelligence. We propose a reinforcement learningalgorithm for chemical process design based on a state-of-the-art actor-criticlogic. Our proposed algorithm represents chemical processes as graphs and usesgraph convolutional neural networks to learn from process graphs. Inparticular, the graph neural networks are implemented within the agentarchitecture to process the states and make decisions. Moreover, we implement ahierarchical and hybrid decision-making process to generate flowsheets, whereunit operations are placed iteratively as discrete decisions and correspondingdesign variables are selected as continuous decisions. We demonstrate thepotential of our method to design economically viable flowsheets in anillustrative case study comprising equilibrium reactions, azeotropicseparation, and recycles. The results show quick learning in discrete,continuous, and hybrid action spaces. Due to the flexible architecture of theproposed reinforcement learning agent, the method is predestined to includelarge action-state spaces and an interface to process simulators in futureresearch.