Reinforcement Learning (RL) controllers have generated excitement within thecontrol community. The primary advantage of RL controllers relative to existingmethods is their ability to optimize uncertain systems independently ofexplicit assumption of process uncertainty. Recent focus on engineeringapplications has been directed towards the development of safe RL controllers.Previous works have proposed approaches to account for constraint satisfactionthrough constraint tightening from the domain of stochastic model predictivecontrol. Here, we extend these approaches to account for plant-model mismatch.Specifically, we propose a data-driven approach that utilizes Gaussianprocesses for the offline simulation model and use the associated posterioruncertainty prediction to account for joint chance constraints and plant-modelmismatch. The method is benchmarked against nonlinear model predictive controlvia case studies. The results demonstrate the ability of the methodology toaccount for process uncertainty, enabling satisfaction of joint chanceconstraints even in the presence of plant-model mismatch.