Drone-based Joint Density Map Estimation, Localization and Tracking with Space-Time Multi-Scale Attention Network

by   Longyin Wen, et al.

This paper proposes a space-time multi-scale attention network (STANet) to solve density map estimation, localization and tracking in dense crowds of video clips captured by drones with arbitrary crowd density, perspective, and flight altitude. Our STANet method aggregates multi-scale feature maps in sequential frames to exploit the temporal coherency, and then predict the density maps, localize the targets, and associate them in crowds simultaneously. A coarse-to-fine process is designed to gradually apply the attention module on the aggregated multi-scale feature maps to enforce the network to exploit the discriminative space-time features for better performance. The whole network is trained in an end-to-end manner with the multi-task loss, formed by three terms, i.e., the density map loss, localization loss and association loss. The non-maximal suppression followed by the min-cost flow framework is used to generate the trajectories of targets' in scenarios. Since existing crowd counting datasets merely focus on crowd counting in static cameras rather than density map estimation, counting and tracking in crowds on drones, we have collected a new large-scale drone-based dataset, DroneCrowd, formed by 112 video clips with 33,600 high resolution frames (i.e., 1920x1080) captured in 70 different scenarios. With intensive amount of effort, our dataset provides 20,800 people trajectories with 4.8 million head annotations and several video-level attributes in sequences. Extensive experiments are conducted on two challenging public datasets, i.e., Shanghaitech and UCF-QNRF, and our DroneCrowd, to demonstrate that STANet achieves favorable performance against the state-of-the-arts. The datasets and codes can be found at https://github.com/VisDrone.


page 3

page 8


Detection, Tracking, and Counting Meets Drones in Crowds: A Benchmark

To promote the developments of object detection, tracking and counting a...

Dual Path Multi-Scale Fusion Networks with Attention for Crowd Counting

The task of crowd counting in varying density scenes is an extremely dif...

A Unified Multi-Task Learning Framework of Real-Time Drone Supervision for Crowd Counting

In this paper, a novel Unified Multi-Task Learning Framework of Real-Tim...

Indirect-Instant Attention Optimization for Crowd Counting in Dense Scenes

One of appealing approaches to guiding learnable parameter optimization,...

Hybrid Graph Neural Networks for Crowd Counting

Crowd counting is an important yet challenging task due to the large sca...

Motion-guided Non-local Spatial-Temporal Network for Video Crowd Counting

We study video crowd counting, which is to estimate the number of object...

Attention to Head Locations for Crowd Counting

Occlusions, complex backgrounds, scale variations and non-uniform distri...

Please sign up or login with your details

Forgot password? Click here to reset