Safe interaction with the environment is one of the most challenging aspectsof Reinforcement Learning (RL) when applied to real-world problems. This isparticularly important when unsafe actions have a high or irreversible negativeimpact on the environment. In the context of network management operations,Remote Electrical Tilt (RET) optimisation is a safety-critical application inwhich exploratory modifications of antenna tilt angles of base stations cancause significant performance degradation in the network. In this paper, wepropose a modular Safe Reinforcement Learning (SRL) architecture which is thenused to address the RET optimisation in cellular networks. In this approach, asafety shield continuously benchmarks the performance of RL agents against safebaselines, and determines safe antenna tilt updates to be performed on thenetwork. Our results demonstrate improved performance of the SRL agent over thebaseline while ensuring the safety of the performed actions.