Abstract
A model used for velocity control during car following was proposed based ondeep reinforcement learning (RL). To fulfil the multi-objectives of carfollowing, a reward function reflecting driving safety, efficiency, and comfortwas constructed. With the reward function, the RL agent learns to controlvehicle speed in a fashion that maximizes cumulative rewards, through trialsand errors in the simulation environment. A total of 1,341 car-following eventsextracted from the Next Generation Simulation (NGSIM) dataset were used totrain the model. Car-following behavior produced by the model were comparedwith that observed in the empirical NGSIM data, to demonstrate the model'sability to follow a lead vehicle safely, efficiently, and comfortably. Resultsshow that the model demonstrates the capability of safe, efficient, andcomfortable velocity control in that it 1) has small percentages (8\%) ofdangerous minimum time to collision values (\textless\ 5s) than human driversin the NGSIM data (35\%); 2) can maintain efficient and safe headways in therange of 1s to 2s; and 3) can follow the lead vehicle comfortably with smoothacceleration. The results indicate that reinforcement learning methods couldcontribute to the development of autonomous driving systems.