Motion-Based Sign Language Video Summarization using Curvature and Torsion

by   Evangelos G. Sartinas, et al.

An interesting problem in many video-based applications is the generation of short synopses by selecting the most informative frames, a procedure which is known as video summarization. For sign language videos the benefits of using the t-parameterized counterpart of the curvature of the 2-D signer's wrist trajectory to identify keyframes, have been recently reported in the literature. In this paper we extend these ideas by modeling the 3-D hand motion that is extracted from each frame of the video. To this end we propose a new informative function based on the t-parameterized curvature and torsion of the 3-D trajectory. The method to characterize video frames as keyframes depends on whether the motion occurs in 2-D or 3-D space. Specifically, in the case of 3-D motion we look for the maxima of the harmonic mean of the curvature and torsion of the target's trajectory; in the planar motion case we seek for the maxima of the trajectory's curvature. The proposed 3-D feature is experimentally evaluated in applications of sign language videos on (1) objective measures using ground-truth keyframe annotations, (2) human-based evaluation of understanding, and (3) gloss classification and the results obtained are promising.


Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses

Recent work have addressed the generation of human poses represented by ...

Hand-tremor frequency estimation in videos

We focus on the problem of estimating human hand-tremor frequency from i...

Trajectory-Based Recognition of Dynamic Persian Sign Language Using Hidden Markov Model

Sign Language Recognition (SLR) is an important step in facilitating the...

Signs in time: Encoding human motion as a temporal image

The goal of this work is to recognise and localise short temporal signal...

A Graph-based Ranking Approach to Extract Key-frames for Static Video Summarization

Video abstraction has become one of the efficient approaches to grasp th...

Extraction of Key-frames of Endoscopic Videos by using Depth Information

A deep learning-based monocular depth estimation (MDE) technique is prop...

Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues

DensePose supersedes traditional landmark detectors by densely mapping i...

Please sign up or login with your details

Forgot password? Click here to reset