Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networks

by   Chengxi Li, et al.

To enable intelligent automated driving systems, a promising strategy is to understand how human drives and interacts with road users in complicated driving situations. In this paper, we propose a 3D-aware egocentric spatial-temporal interaction framework for automated driving applications. Graph convolution networks (GCN) is devised for interaction modeling. We introduce three novel concepts into GCN. First, we decompose egocentric interactions into ego-thing and ego-stuff interaction, modeled by two GCNs. In both GCNs, ego nodes are introduced to encode the interaction between thing objects (e.g., car and pedestrian), and interaction between stuff objects (e.g., lane marking and traffic light). Second, objects' 3D locations are explicitly incorporated into GCN to better model egocentric interactions. Third, to implement ego-stuff interaction in GCN, we propose a MaskAlign operation to extract features for irregular objects. We validate the proposed framework on tactical driver behavior recognition. Extensive experiments are conducted using Honda Research Institute Driving Dataset, the largest dataset with diverse tactical driver behavior annotations. Our framework demonstrates substantial performance boost over baselines on the two experimental settings by 3.9 visualize the learned affinity matrices, which encode ego-thing and ego-stuff interactions, to showcase the proposed framework can capture interactions effectively.


page 1

page 3

page 6


RAIST: Learning Risk Aware Traffic Interactions via Spatio-Temporal Graph Convolutional Networks

A key aspect of driving a road vehicle is to interact with the other roa...

Interaction Graphs for Object Importance Estimation in On-road Driving Videos

A vehicle driving along the road is surrounded by many objects, but only...

Spatial-Temporal Gating-Adjacency GCN for Human Motion Prediction

Predicting future motion based on historical motion sequence is a fundam...

Graph-Based Interaction-Aware Multimodal 2D Vehicle Trajectory Prediction using Diffusion Graph Convolutional Networks

Predicting vehicle trajectories is crucial for ensuring automated vehicl...

Dynamic Prediction of Origin-Destination Flows Using Fusion Line Graph Convolutional Networks

Modern intelligent transportation systems provide data that allow real-t...

IDE-Net: Interactive Driving Event and Pattern Extraction from Human Data

Autonomous vehicles (AVs) need to share the road with multiple, heteroge...

Towards Accurate Vehicle Behaviour Classification With Multi-Relational Graph Convolutional Networks

Understanding on-road vehicle behaviour from a temporal sequence of sens...

Please sign up or login with your details

Forgot password? Click here to reset