Abstract
Cooperative Adaptive Cruise Control (CACC) plays a pivotal role in enhancingtraffic efficiency and safety in Connected and Autonomous Vehicles (CAVs).Reinforcement Learning (RL) has proven effective in optimizing complexdecision-making processes in CACC, leading to improved system performance andadaptability. Among RL approaches, Multi-Agent Reinforcement Learning (MARL)has shown remarkable potential by enabling coordinated actions among multipleCAVs through Centralized Training with Decentralized Execution (CTDE). However,MARL often faces scalability issues, particularly when CACC vehicles suddenlyjoin or leave the platoon, resulting in performance degradation. To addressthese challenges, we propose Communication-Aware Reinforcement Learning(CA-RL). CA-RL includes a communication-aware module that extracts andcompresses vehicle communication information through forward and backwardinformation transmission modules. This enables efficient cyclic informationpropagation within the CACC traffic flow, ensuring policy consistency andmitigating the scalability problems of MARL in CACC. Experimental resultsdemonstrate that CA-RL significantly outperforms baseline methods in varioustraffic scenarios, achieving superior scalability, robustness, and overallsystem performance while maintaining reliable performance despite changes inthe number of participating vehicles.