Decoupling Object Detection from Human-Object Interaction Recognition

12/13/2021
by   Ying Jin, et al.
0

We propose DEFR, a DEtection-FRee method to recognize Human-Object Interactions (HOI) at image level without using object location or human pose. This is challenging as the detector is an integral part of existing methods. In this paper, we propose two findings to boost the performance of the detection-free approach, which significantly outperforms the detection-assisted state of the arts. Firstly, we find it crucial to effectively leverage the semantic correlations among HOI classes. Remarkable gain can be achieved by using language embeddings of HOI labels to initialize the linear classifier, which encodes the structure of HOIs to guide training. Further, we propose Log-Sum-Exp Sign (LSE-Sign) loss to facilitate multi-label learning on a long-tailed dataset by balancing gradients over all classes in a softmax format. Our detection-free approach achieves 65.6 mAP in HOI classification on HICO, outperforming the detection-assisted state of the art (SOTA) by 18.5 mAP, and 52.7 mAP in one-shot classes, surpassing the SOTA by 27.3 mAP. Different from previous work, our classification model (DEFR) can be directly used in HOI detection without any additional training, by connecting to an off-the-shelf object detector whose bounding box output is converted to binary masks for DEFR. Surprisingly, such a simple connection of two decoupled models achieves SOTA performance (32.35 mAP).

READ FULL TEXT

page 4

page 8

research
03/10/2022

The Overlooked Classifier in Human-Object Interaction Recognition

Human-Object Interaction (HOI) recognition is challenging due to two fac...
research
07/27/2021

Is Object Detection Necessary for Human-Object Interaction Recognition?

This paper revisits human-object interaction (HOI) recognition at image ...
research
10/02/2019

DeepMark: One-Shot Clothing Detection

The one-shot approach, DeepMark, for fast clothing detection as a modifi...
research
11/21/2021

FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection

Existing anchor-base oriented object detection methods have achieved ama...
research
09/09/2021

ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection

A common problem in the task of human-object interaction (HOI) detection...
research
07/17/2020

Detecting Human-Object Interactions with Action Co-occurrence Priors

A common problem in human-object interaction (HOI) detection task is tha...
research
06/10/2020

Diagnosing Rarity in Human-Object Interaction Detection

Human-object interaction (HOI) detection is a core task in computer visi...

Please sign up or login with your details

Forgot password? Click here to reset