Body movement to sound interface with vector autoregressive hierarchical hidden Markov models

by   Dimitrije Marković, et al.

Interfacing a kinetic action of a person to an action of a machine system is an important research topic in many application areas. One of the key factors for intimate human-machine interaction is the ability of the control algorithm to detect and classify different user commands with shortest possible latency, thus making a highly correlated link between cause and effect. In our research, we focused on the task of mapping user kinematic actions into sound samples. The presented methodology relies on the wireless sensor nodes equipped with inertial measurement units and the real-time algorithm dedicated for early detection and classification of a variety of movements/gestures performed by a user. The core algorithm is based on the approximate Bayesian inference of Vector Autoregressive Hierarchical Hidden Markov Models (VAR-HHMM), where models database is derived from the set of motion gestures. The performance of the algorithm was compared with an online version of the K-nearest neighbours (KNN) algorithm, where we used offline expert based classification as the benchmark. In almost all of the evaluation metrics (e.g. confusion matrix, recall and precision scores) the VAR-HHMM algorithm outperformed KNN. Furthermore, the VAR-HHMM algorithm, in some cases, achieved faster movement onset detection compared with the offline standard. The proposed concept, although envisioned for movement-to-sound application, could be implemented in other human-machine interfaces.


page 2

page 7

page 8

page 9


Ballroom Dance Movement Recognition Using a Smart Watch

Inertial Measurement Unit (IMU) sensors are being increasingly used to d...

HMM and DTW for evaluation of therapeutical gestures using kinect

Automatic recognition of the quality of movement in human beings is a ch...

Affective Movement Generation using Laban Effort and Shape and Hidden Markov Models

Body movements are an important communication medium through which affec...

Fast Gesture Recognition with Multiple Stream Discrete HMMs on 3D Skeletons

HMMs are widely used in action and gesture recognition due to their impl...

Visual Rendering of Shapes on 2D Display Devices Guided by Hand Gestures

Designing of touchless user interface is gaining popularity in various c...

Evidence Feed Forward Hidden Markov Model: A New Type of Hidden Markov Model

The ability to predict the intentions of people based solely on their vi...

Designing Just-in-Time Detection for Gamified Fitness Frameworks

This paper presents our findings from a multi-year effort to detect moti...

Please sign up or login with your details

Forgot password? Click here to reset