Human-to-Human Interaction Detection

by   Zhenhua Wang, et al.

A comprehensive understanding of interested human-to-human interactions in video streams, such as queuing, handshaking, fighting and chasing, is of immense importance to the surveillance of public security in regions like campuses, squares and parks. Different from conventional human interaction recognition, which uses choreographed videos as inputs, neglects concurrent interactive groups, and performs detection and recognition in separate stages, we introduce a new task named human-to-human interaction detection (HID). HID devotes to detecting subjects, recognizing person-wise actions, and grouping people according to their interactive relations, in one model. First, based on the popular AVA dataset created for action detection, we establish a new HID benchmark, termed AVA-Interaction (AVA-I), by adding annotations on interactive relations in a frame-by-frame manner. AVA-I consists of 85,254 frames and 86,338 interactive groups, and each image includes up to 4 concurrent interactive groups. Second, we present a novel baseline approach SaMFormer for HID, containing a visual feature extractor, a split stage which leverages a Transformer-based model to decode action instances and interactive groups, and a merging stage which reconstructs the relationship between instances and groups. All SaMFormer components are jointly trained in an end-to-end manner. Extensive experiments on AVA-I validate the superiority of SaMFormer over representative methods. The dataset and code will be made public to encourage more follow-up studies.


page 1

page 4

page 5

page 6

page 9


RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection

Human-Object Interaction (HOI) detection devotes to learn how humans int...

Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection

The key of Human-Object Interaction(HOI) recognition is to infer the rel...

IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition

Human interaction recognition is very important in many applications. On...

LAGNet: Logic-Aware Graph Network for Human Interaction Understanding

Compared with the progress made on human activity classification, much l...

A Graph-based Interactive Reasoning for Human-Object Interaction Detection

Human-Object Interaction (HOI) detection devotes to learn how humans int...

Online Localization and Prediction of Actions and Interactions

This paper proposes a person-centric and online approach to the challeng...

Real-Time Cattle Interaction Recognition via Triple-stream Network

In stockbreeding of beef cattle, computer vision-based approaches have b...

Please sign up or login with your details

Forgot password? Click here to reset