Abstract
Large-scale infrastructure systems are crucial for societal welfare, andtheir effective management requires strategic forecasting and interventionmethods that account for various complexities. Our study addresses twochallenges within the Prognostics and Health Management (PHM) framework appliedto sewer assets: modeling pipe degradation across severity levels anddeveloping effective maintenance policies. We employ Multi-State DegradationModels (MSDM) to represent the stochastic degradation process in sewer pipesand use Deep Reinforcement Learning (DRL) to devise maintenance strategies. Acase study of a Dutch sewer network exemplifies our methodology. Our findingsdemonstrate the model's effectiveness in generating intelligent, cost-savingmaintenance strategies that surpass heuristics. It adapts its managementstrategy based on the pipe's age, opting for a passive approach for newer pipesand transitioning to active strategies for older ones to prevent failures andreduce costs. This research highlights DRL's potential in optimizingmaintenance policies. Future research will aim improve the model byincorporating partial observability, exploring various reinforcement learningalgorithms, and extending this methodology to comprehensive infrastructuremanagement.