Abstract
Reinforcement learning (RL) is widely used in autonomous driving tasks andtraining RL models typically involves in a multi-step process: pre-training RLmodels on simulators, uploading the pre-trained model to real-life robots, andfine-tuning the weight parameters on robot vehicles. This sequential process isextremely time-consuming and more importantly, knowledge from the fine-tunedmodel stays local and can not be re-used or leveraged collaboratively. Totackle this problem, we present an online federated RL transfer process forreal-time knowledge extraction where all the participant agents makecorresponding actions with the knowledge learned by others, even when they areacting in very different environments. To validate the effectiveness of theproposed approach, we constructed a real-life collision avoidance system withMicrosoft Airsim simulator and NVIDIA JetsonTX2 car agents, which cooperativelylearn from scratch to avoid collisions in indoor environment with obstacleobjects. We demonstrate that with the proposed framework, the simulator caragents can transfer knowledge to the RC cars in real-time, with 27% increase inthe average distance with obstacles and 42% decrease in the collision counts.