Abstract
Deep Reinforcement Learning (DRL) is employed to develop autonomouslyoptimized and custom-designed heat-treatment processes that are both,microstructure-sensitive and energy efficient. Different from conventionalsupervised machine learning, DRL does not rely on static neural networktraining from data alone, but a learning agent autonomously develops optimalsolutions, based on reward and penalty elements, with reduced or nosupervision. In our approach, a temperature-dependent Allen-Cahn model forphase transformation is used as the environment for the DRL agent, serving asthe model world in which it gains experience and takes autonomous decisions.The agent of the DRL algorithm is controlling the temperature of the system, asa model furnace for heat-treatment of alloys. Microstructure goals are definedfor the agent based on the desired microstructure of the phases. Aftertraining, the agent can generate temperature-time profiles for a variety ofinitial microstructure states to reach the final desired microstructure state.The agent's performance and the physical meaning of the heat-treatment profilesgenerated are investigated in detail. In particular, the agent is capable ofcontrolling the temperature to reach the desired microstructure starting from avariety of initial conditions. This capability of the agent in handling avariety of conditions paves the way for using such an approach also forrecycling-oriented heat treatment process design where the initial compositioncan vary from batch to batch, due to impurity intrusion, and also for thedesign of energy-efficient heat treatments. For testing this hypothesis, anagent without penalty on the total consumed energy is compared with one thatconsiders energy costs. The energy cost penalty is imposed as an additionalcriterion on the agent for finding the optimal temperature-time profile.