Tactile information is important for robust performance in robotic tasks thatinvolve physical interaction, such as object manipulation. However, with moredata included in the reasoning and control process, modeling behavior becomesincreasingly difficult. Deep Reinforcement Learning (DRL) produced promisingresults for learning complex behavior in various domains, includingtactile-based manipulation in robotics. In this work, we present ouropen-source reinforcement learning environments for the TIAGo service robot.They produce tactile sensor measurements that resemble those of a realsensorised gripper for TIAGo, encouraging research in transfer learning of DRLpolicies. Lastly, we show preliminary training results of a learned forcecontrol policy and compare it to a classical PI controller.