Modeling Label Semantics Improves Activity Recognition

by   Xiyuan Zhang, et al.

Human activity recognition (HAR) aims to classify sensory time series into different activities, with wide applications in activity tracking, healthcare, human computer interaction, etc. Existing HAR works improve recognition performance by designing more complicated feature extraction methods, but they neglect the label semantics by simply treating labels as integer IDs. We find that many activities in the current HAR datasets have shared label names, e.g., "open door" and "open fridge", "walk upstairs" and "walk downstairs". Through some exploratory analysis, we find that such shared structure in activity names also maps to similarity in the input features. To this end, we design a sequence-to-sequence framework to decode the label name semantics rather than classifying labels as integer IDs. Our proposed method decomposes learning activities into learning shared tokens ("open", "walk"), which is easier than learning the joint distribution ("open fridge", "walk upstairs") and helps transfer learning to activities with insufficient data samples. For datasets originally without shared tokens in label names, we also offer an automated method, using OpenAI's ChatGPT, to generate shared actions and objects. Extensive experiments on seven HAR benchmark datasets demonstrate the state-of-the-art performance of our method. We also show better performance in the long-tail activity distribution settings and few-shot settings.


page 1

page 2

page 3

page 4


Multi-Label Activity Recognition using Activity-specific Features

We introduce an approach to multi-label activity recognition by extracti...

Tutorial on Deep Learning for Human Activity Recognition

Activity recognition systems that are capable of estimating human activi...

A Tree-structure Convolutional Neural Network for Temporal Features Exaction on Sensor-based Multi-resident Activity Recognition

With the propagation of sensor devices applied in smart home, activity r...

Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors

Wearables are fundamental to improving our understanding of human activi...

Unsupervised Human Activity Recognition through Two-stage Prompting with ChatGPT

Wearable sensor devices, which offer the advantage of recording daily ob...

Usual and Unusual Human Activity Recognition in Video using Deep Learning and Artificial Intelligence for Security Applications

The main objective of Human Activity Recognition (HAR) is to detect vari...

Time Series Segmentation through Automatic Feature Learning

Internet of things (IoT) applications have become increasingly popular i...

Please sign up or login with your details

Forgot password? Click here to reset