Abstract
Defects during production may lead to material waste, which is a significantchallenge for many companies as it reduces revenue and negatively impactssustainability and the environment. An essential reason for material waste is alow degree of automation, especially in industries that currently have a lowdegree of digitalization, such as steel forging. Those industries typicallyrely on heavy and old machinery such as large induction ovens that are mostlycontrolled manually or using well-known recipes created by experts. However,standard recipes may fail when unforeseen events happen, such as an unplannedstop in production, which may lead to overheating and thus material degradationduring the forging process. In this paper, we develop a digital twin-basedoptimization strategy for the heating process for a forging line to automatethe development of an optimal control policy that adjusts the power for theheating coils in an induction oven based on temperature data observed frompyrometers. We design a digital twin-based deep reinforcement learning (DTRL)framework and train two different deep reinforcement learning (DRL) models forthe heating phase using a digital twin of the forging line. The twin is basedon a simulator that contains a heating transfer and movement model, which isused as an environment for the DRL training. Our evaluation shows that bothmodels significantly reduce the temperature unevenness and can help to automatethe traditional heating process.