An Autonomous Mobility on Demand Platform Supporting Cooperative Driving with Visual Context Generation for LMMs

As Connected Autonomous Vehicles (CAVs) emerge as a cornerstone of future transportation systems, Autonomous Mobility on Demand (AMoD) platforms stand to revolutionize urban mobility by mitigating traffic congestion and enhancing passenger comfort and convenience. However, existing research primarily addresses either vehicle dispatch using real-world datasets to estimate the time cost for each CAV to its destination, or emphasizes cooperative motion planning of the CAVs for obstacle avoidance. Moreover, these studies often overlook the integration of the following two aspects: CAV destination assignments and cooperative motion planning in AMoD systems. To bridge this gap, our proposed CoDriveVis offers an integrated platform that integrates microscopic motion planning and control with macroscopic vehicle scheduling and dispatching. CoDriveVis leverages CARLA’s perception capabilities to produce multimodal outputs such as LiDAR-generated point clouds, images, and videos from diverse perspectives, which can inform traffic system operations, especially when coupled with large multimodal models (LMMs). Additionally, it features the Bird’s Eye View (BEV) image generation to visualize vehicle dispatching and request distribution across the map. The scheduling integration of the platform is streamlined through organized dictionaries for CAVs and passenger requests, while its control modes support both position transition and throttle/steering inputs. CoDriveVis also facilitates destination changes for updating the coordination of CAVs. Experimental deployment of representative methods within CoDriveVis demonstrates its efficacy and practicality as a robust simulation platform for advancing AMoD systems. The code is available at https://github.com/henryhcliu/CoDriveVis.git.

Recommended citation: H. Liu, H. Liu, and J. Ma, “An Autonomous Mobility on Demand Platform Supporting Cooperative Driving with Visual Context Generation for LMMs,” in Proceedings of IEEE International Conference on Intelligent Transportation Systems (ITSC 2025), IEEE, 2025, pp. 1-8. TBA