Online Learnable Keyframe Extraction in Videos and its Application with Semantic Word Vector in Action Recognition

by   G. M. Mashrur-E-Elahi, et al.

Video processing has become a popular research direction in computer vision due to its various applications such as video summarization, action recognition, etc. Recently, deep learning-based methods have achieved impressive results in action recognition. However, these methods need to process a full video sequence to recognize the action, even though most of these frames are similar and non-essential to recognizing a particular action. Additionally, these non-essential frames increase the computational cost and can confuse a method in action recognition. Instead, the important frames called keyframes not only are helpful in the recognition of an action but also can reduce the processing time of each video sequence for classification or in other applications, e.g. summarization. As well, current methods in video processing have not yet been demonstrated in an online fashion. Motivated by the above, we propose an online learnable module for keyframe extraction. This module can be used to select key-shots in video and thus can be applied to video summarization. The extracted keyframes can be used as input to any deep learning-based classification model to recognize action. We also propose a plugin module to use the semantic word vector as input along with keyframes and a novel train/test strategy for the classification models. To our best knowledge, this is the first time such an online module and train/test strategy have been proposed. The experimental results on many commonly used datasets in video summarization and in action recognition have shown impressive results using the proposed module.


page 1

page 2

page 3

page 4


Video-based Human Action Recognition using Deep Learning: A Review

Human action recognition is an important application domain in computer ...

Online Action Recognition based on Incremental Learning of Weighted Covariance Descriptors

Different from traditional action recognition based on video segments, o...

Mobile Video Action Recognition

Video action recognition, which is topical in computer vision and video ...

No frame left behind: Full Video Action Recognition

Not all video frames are equally informative for recognizing an action. ...

Curvature: A signature for Action Recognition in Video Sequences

In this paper, a novel signature of human action recognition, namely the...

Selective Volume Mixup for Video Action Recognition

The recent advances in Convolutional Neural Networks (CNNs) and Vision T...

Effective Action Recognition with Embedded Key Point Shifts

Temporal feature extraction is an essential technique in video-based act...

Please sign up or login with your details

Forgot password? Click here to reset