Abstract
We provide a simple hardware wrapper around the Quanser'shardware-in-the-loop software development kit (HIL SDK) to allow for easydevelopment of new Quanser hardware. To connect to the hardware we use a modulewritten in Cython. The internal QuanserWrapper class handles most of thedifficult aspects of interacting with hardware, including the timing (using ahardware timer), and ensuring the data sent to hardware is safe and correct,where safety corresponds to safe operating voltage and current for thespecified hardware. Much of the recent success of Reinforcement learning (RL)has been made possible with training and testing tools like OpenAI Gym andDeepmind Control Suite. Unfortunately, tools for quickly testing andtransferring high-frequency RL algorithms from simulation to real hardwareenvironment remain mostly absent. We present Blue River Controls, a tool thatallows to train and test reinforcement learning algorithms on real-worldhardware. It features a simple interface based on OpenAI Gym, that worksdirectly on both simulation and hardware. We use Quanser's Qube Servo2-USBplatform, an underactuated rotary pendulum as an initial testing device. Wealso provide tools to simplify training RL algorithms on other hardware.Several baselines, from both classical controllers and pretrained RL agents areincluded to compare performance across tasks. Blue River Controls is availableat this https URL: https://github.com/BlueRiverTech/quanser-openai-driver