Action Recognition with Visual Attention on Skeleton Images

by   Zhengyuan Yang, et al.

Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages and the joint predictions can be inaccurate. To solve the two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree traversal order, which enhances the semantic meaning of skeleton images and better preserves the structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced to illustrate the two-branch attention architecture. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Experiment results on the NTU RGB+D dataset and the SBU Kinetic Interaction dataset show that our proposed approach outperforms the state-of-the-art, as well as the effectiveness of each component.


page 1

page 2

page 3

page 4


Action Recognition with Domain Invariant Features of Skeleton Image

Due to the fast processing-speed and robustness it can achieve, skeleton...

Investigation of Different Skeleton Features for CNN-based 3D Action Recognition

Deep learning techniques are being used in skeleton based action recogni...

Skepxels: Spatio-temporal Image Representation of Human Skeleton Joints for Action Recognition

Human skeleton joints are popular for action analysis since they can be ...

Action Capsules: Human Skeleton Action Recognition

Due to the compact and rich high-level representations offered, skeleton...

Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Skeleton-based action recognition has made great progress recently, but ...

Tensor Representations via Kernel Linearization for Action Recognition from 3D Skeletons (Extended Version)

In this paper, we explore tensor representations that can compactly capt...

Skeleton-based action analysis for ADHD diagnosis

Attention Deficit Hyperactivity Disorder (ADHD) is a common neurobehavio...

Please sign up or login with your details

Forgot password? Click here to reset