Abstract
Reinforcement Learning (RL) has shown excellent performance in solvingdecision-making and control problems of autonomous driving, which isincreasingly applied in diverse driving scenarios. However, driving is amulti-attribute problem, leading to challenges in achieving multi-objectivecompatibility for current RL methods, especially in both policy execution andpolicy iteration. On the one hand, the common action space structure withsingle action type limits driving flexibility or results in large behaviorfluctuations during policy execution. On the other hand, the multi-attributeweighted single reward function result in the agent's disproportionateattention to certain objectives during policy iterations. To this end, wepropose a Multi-objective Ensemble-Critic reinforcement learning method withHybrid Parametrized Action for multi-objective compatible autonomous driving.Specifically, a parameterized action space is constructed to generate hybriddriving actions, combining both abstract guidance and concrete controlcommands. A multi-objective critics architecture is constructed consideringmultiple attribute rewards, to ensure simultaneously focusing on differentdriving objectives. Additionally, uncertainty-based exploration strategy isintroduced to help the agent faster approach viable driving policy. Theexperimental results in both the simulated traffic environment and the HighDdataset demonstrate that our method can achieve multi-objective compatibleautonomous driving in terms of driving efficiency, action consistency, andsafety. It enhances the general performance of the driving while significantlyincreasing training efficiency.