Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approach

by   Zihan Guo, et al.

The recent proliferation of the research on multi-agent deep reinforcement learning (MDRL) offers an encouraging way to coordinate multiple connected and automated vehicles (CAVs) to pass the intersection. In this paper, we apply a value decomposition-based MDRL approach (QMIX) to control various CAVs in mixed-autonomy traffic of different densities to efficiently and safely pass the non-signalized intersection with fairish fuel consumption. Implementation tricks including network-level improvements, Q value update by TD (λ), and reward clipping operation are added to the pure QMIX framework, which is expected to improve the convergence speed and the asymptotic performance of the original version. The efficacy of our approach is demonstrated by several evaluation metrics: average speed, the number of collisions, and average fuel consumption per episode. The experimental results show that our approach's convergence speed and asymptotic performance can exceed that of the original QMIX and the proximal policy optimization (PPO), a state-of-the-art reinforcement learning baseline applied to the non-signalized intersection. Moreover, CAVs under the lower traffic flow controlled by our method can improve their average speed without collisions and consume the least fuel. The training is additionally conducted under the doubled traffic density, where the learning reward converges. Consequently, the model with maximal reward and minimum crashes can still guarantee low fuel consumption, but slightly reduce the efficiency of vehicles and induce more collisions than the lower-traffic counterpart, implying the difficulty of generalizing RL policy to more advanced scenarios.


page 1

page 3

page 7

page 8


Courteous Behavior of Automated Vehicles at Unsignalized Intersections via Reinforcement Learning

The transition from today's mostly human-driven traffic to a purely auto...

CVLight: Deep Reinforcement Learning for Adaptive Traffic Signal Control with Connected Vehicles

This paper develops a reinforcement learning (RL) scheme for adaptive tr...

Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected and Automated Vehicles at Signalized Intersections

Taking advantage of both vehicle-to-everything (V2X) communication and a...

Centralized Conflict-free Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization

Connected vehicles will change the modes of future transportation manage...

Learning to Control and Coordinate Hybrid Traffic Through Robot Vehicles at Complex and Unsignalized Intersections

Intersections are essential road infrastructures for traffic in modern m...

A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities

Connected and automated vehicles (CAVs) can alleviate traffic congestion...

Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles – A Deep Reinforcement Learning Approach

Stop-and-go traffic poses many challenges to tranportation system, but i...

Please sign up or login with your details

Forgot password? Click here to reset