Event Transformer

by   Zhihao Li, et al.

The event camera is a bio-vision inspired camera with high dynamic range, high response speed, and low power consumption, recently attracting extensive attention for its use in vast vision tasks. Unlike the conventional cameras that output intensity frame at a fixed time interval, event camera records the pixel brightness change (a.k.a., event) asynchronously (in time) and sparsely (in space). Existing methods often aggregate events occurred in a predefined temporal duration for downstream tasks, which apparently overlook varying behaviors of fine-grained temporal events. This work proposes the Event Transformer to directly process the event sequence in its native vectorized tensor format. It cascades a Local Transformer (LXformer) for exploiting the local temporal correlation, a Sparse Conformer (SCformer) for embedding the local spatial similarity, and a Global Transformer (GXformer) for further aggregating the global information in a serial means to effectively characterize the time and space correlations from input raw events for the generation of effective spatiotemporal features used for tasks. LXformer and SCformer, Experimental studies have been extensively conducted in comparison to another fourteen existing algorithms upon five different datasets widely used for classification. Quantitative results report the state-of-the-arts classification accuracy and the least computational resource requirements, of the Event Transformer, making it practically attractive for event-based vision tasks.


page 1

page 2

page 3

page 4


AET-EFN: A Versatile Design for Static and Dynamic Event-Based Vision

The neuromorphic event cameras, which capture the optical changes of a s...

Exploiting Spatial Sparsity for Event Cameras with Visual Transformers

Event cameras report local changes of brightness through an asynchronous...

Temporal Up-Sampling for Asynchronous Events

The event camera is a novel bio-inspired vision sensor. When the brightn...

Event Transformer. A sparse-aware solution for efficient event data processing

Event cameras are sensors of great interest for many applications that r...

Tracking Particles Ejected From Active Asteroid Bennu With Event-Based Vision

Early detection and tracking of ejecta in the vicinity of small solar sy...

Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers

Neuromorphic vision is a bio-inspired technology that has triggered a pa...

Improved Regularization of Event-based Learning by Reversing and Drifting

Event camera has an enormous potential in challenging scenes for its adv...

Please sign up or login with your details

Forgot password? Click here to reset