CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

04/10/2017
by   Tianmin Shu, et al.
0

This work is about recognizing human activities occurring in videos at distinct semantic levels, including individual actions, interactions, and group activities. The recognition is realized using a two-level hierarchy of Long Short-Term Memory (LSTM) networks, forming a feed-forward deep architecture, which can be trained end-to-end. In comparison with existing architectures of LSTMs, we make two key contributions giving the name to our approach as Confidence-Energy Recurrent Network -- CERN. First, instead of using the common softmax layer for prediction, we specify a novel energy layer (EL) for estimating the energy of our predictions. Second, rather than finding the common minimum-energy class assignment, which may be numerically unstable under uncertainty, we specify that the EL additionally computes the p-values of the solutions, and in this way estimates the most confident energy minimum. The evaluation on the Collective Activity and Volleyball datasets demonstrates: (i) advantages of our two contributions relative to the common softmax and energy-minimization formulations and (ii) a superior performance relative to the state-of-the-art approaches.

READ FULL TEXT

page 3

page 8

research
11/28/2016

Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition

We present a unified framework for understanding human social behaviors ...
research
04/05/2019

Convolutional Relational Machine for Group Activity Recognition

We present an end-to-end deep Convolutional Neural Network called Convol...
research
07/12/2022

Hunting Group Clues with Transformers for Social Group Activity Recognition

This paper presents a novel framework for social group activity recognit...
research
05/26/2016

Learning Latent Sub-events in Activity Videos Using Temporal Attention Filters

In this paper, we newly introduce the concept of temporal attention filt...
research
03/28/2017

Ensembles of Deep LSTM Learners for Activity Recognition using Wearables

Recently, deep learning (DL) methods have been introduced very successfu...
research
12/05/2015

A Deep Structured Model with Radius-Margin Bound for 3D Human Activity Recognition

Understanding human activity is very challenging even with the recently ...

Please sign up or login with your details

Forgot password? Click here to reset