Abstract
Vehicle-to-vehicle (V2V) cooperative autonomous driving holds great promisefor improving safety by addressing the perception and prediction uncertaintiesinherent in single-agent systems. However, traditional cooperative methods areconstrained by rigid collaboration protocols and limited generalization tounseen interactive scenarios. While LLM-based approaches offer generalizedreasoning capabilities, their challenges in spatial planning and unstableinference latency hinder their direct application in cooperative driving. Toaddress these limitations, we propose CoLMDriver, the first full-pipelineLLM-based cooperative driving system, enabling effective language-basednegotiation and real-time driving control. CoLMDriver features a paralleldriving pipeline with two key components: (i) an LLM-based negotiation moduleunder an actor-critic paradigm, which continuously refines cooperation policiesthrough feedback from previous decisions of all vehicles; and (ii) anintention-guided waypoint generator, which translates negotiation outcomes intoexecutable waypoints. Additionally, we introduce InterDrive, a CARLA-basedsimulation benchmark comprising 10 challenging interactive driving scenariosfor evaluating V2V cooperation. Experimental results demonstrate thatCoLMDriver significantly outperforms existing approaches, achieving an 11%higher success rate across diverse highly interactive V2V driving scenarios.Code will be released on https://github.com/cxliu0314/CoLMDriver.