Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation

With the advance of technologies, machines are increasingly present in people's daily lives. Thus, there has been more and more effort for developing interfaces, such as dynamic gestures, that provide an intuitive way of interaction. Currently, the most common trend is to use multimodal data, as depth and skeleton information, to try to recognize dynamic gestures. However, the use of only color information would be more interesting, once RGB cameras are usually found in almost every public place, and could be used for gesture recognition without the need to install other equipment. The main problem with this approach is the difficulty of representing spatio-temporal information using just color. With this in mind, we propose a technique that we called Star RGB, capable of describing a videoclip containing a dynamic gesture as an RGB image. This image is then passed to a classifier formed by two Resnet CNN's, a soft-attention ensemble, and a multilayer perceptron, which returns the predicted class label that indicates to which type of gesture the input video belongs. Experiments were carried out using the Montalbano and GRIT datasets. On the Montalbano dataset, the proposed approach achieved an accuracy of 94.58 considering only color information. On the GRIT dataset, our proposal achieves more than 98 reference approach in more than 6


page 6

page 7

page 9

page 10

page 14


3D dynamic hand gestures recognition using the Leap Motion sensor and convolutional neural networks

Defining methods for the automatic understanding of gestures is of param...

CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

Human activity and gesture recognition is an important component of rapi...

Gesture Recognition in RGB Videos UsingHuman Body Keypoints and Dynamic Time Warping

Gesture recognition opens up new ways for humans to intuitively interact...

Statistical and Spatio-temporal Hand Gesture Features for Sign Language Recognition using the Leap Motion Sensor

In modern society, people should not be identified based on their disabi...

SynthoGestures: A Novel Framework for Synthetic Dynamic Hand Gesture Generation for Driving Scenarios

Creating a diverse and comprehensive dataset of hand gestures for dynami...

A Spatio-Temporal Multilayer Perceptron for Gesture Recognition

Gesture recognition is essential for the interaction of autonomous vehic...

ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly

Deep learning, based on which many modern algorithms operate, is well kn...

Please sign up or login with your details

Forgot password? Click here to reset