Hand gesture recognition using 802.11ad mmWave sensor in the mobile device

by   Yuwei Ren, et al.

We explore the feasibility of AI assisted hand-gesture recognition using 802.11ad 60GHz (mmWave) technology in smartphones. Range-Doppler information (RDI) is obtained by using pulse Doppler radar for gesture recognition. We built a prototype system, where radar sensing and WLAN communication waveform can coexist by time-division duplex (TDD), to demonstrate the real-time hand-gesture inference. It can gather sensing data and predict gestures within 100 milliseconds. First, we build the pipeline for the real-time feature processing, which is robust to occasional frame drops in the data stream. RDI sequence restoration is implemented to handle the frame dropping in the continuous data stream, and also applied to data augmentation. Second, different gestures RDI are analyzed, where finger and hand motions can clearly show distinctive features. Third, five typical gestures (swipe, palm-holding, pull-push, finger-sliding and noise) are experimented with, and a classification framework is explored to segment the different gestures in the continuous gesture sequence with arbitrary inputs. We evaluate our architecture on a large multi-person dataset and report > 95 model. Further, a pure CNN model is developed to fit to on-device implementation, which minimizes the inference latency, power consumption and computation cost. And the accuracy of this CNN model is more than 93 2.29K parameters.


page 1

page 3

page 4


Doppler-Radar Based Hand Gesture Recognition System Using Convolutional Neural Networks

Hand gesture recognition has long been a hot topic in human computer int...

Micro Hand Gesture Recognition System Using Ultrasonic Active Sensing

In this paper, we propose a micro hand gesture recognition system using ...

Eat-Radar: Continuous Fine-Grained Eating Gesture Detection Using FMCW Radar and 3D Temporal Convolutional Network

Unhealthy dietary habits are considered as the primary cause of multiple...

SolarGest: Ubiquitous and Battery-free Gesture Recognition using Solar Cells

We design a system, SolarGest, which can recognize hand gestures near a ...

Robust and customized methods for real-time hand gesture recognition under object-occlusion

Dynamic hand tracking and gesture recognition is a hard task since there...

Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds

We present Tesla-Rapture, a gesture recognition interface for point clou...

Real-time on-device nod and shake recognition

We discuss methods for teaching systems to identify gestures such as hea...

Please sign up or login with your details

Forgot password? Click here to reset