Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation

by   Chaofan Zheng, et al.

The current studies of Scene Graph Generation (SGG) focus on solving the long-tailed problem for generating unbiased scene graphs. However, most de-biasing methods overemphasize the tail predicates and underestimate head ones throughout training, thereby wrecking the representation ability of head predicate features. Furthermore, these impaired features from head predicates harm the learning of tail predicates. In fact, the inference of tail predicates heavily depends on the general patterns learned from head ones, e.g., "standing on" depends on "on". Thus, these de-biasing SGG methods can neither achieve excellent performance on tail predicates nor satisfying behaviors on head ones. To address this issue, we propose a Dual-branch Hybrid Learning network (DHL) to take care of both head predicates and tail ones for SGG, including a Coarse-grained Learning Branch (CLB) and a Fine-grained Learning Branch (FLB). Specifically, the CLB is responsible for learning expertise and robust features of head predicates, while the FLB is expected to predict informative tail predicates. Furthermore, DHL is equipped with a Branch Curriculum Schedule (BCS) to make the two branches work well together. Experiments show that our approach achieves a new state-of-the-art performance on VG and GQA datasets and makes a trade-off between the performance of tail predicates and head ones. Moreover, extensive experiments on two downstream tasks (i.e., Image Captioning and Sentence-to-Graph Retrieval) further verify the generalization and practicability of our method.


page 1

page 4

page 11


Learning To Generate Scene Graph from Head to Tail

Scene Graph Generation (SGG) represents objects and their interactions w...

Adaptive Fine-Grained Predicates Learning for Scene Graph Generation

The performance of current Scene Graph Generation (SGG) models is severe...

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Today, scene graph generation(SGG) task is largely limited in realistic ...

Fine-Grained Predicates Learning for Scene Graph Generation

The performance of current Scene Graph Generation models is severely ham...

Unbiased Scene Graph Generation via Two-stage Causal Modeling

Despite the impressive performance of recent unbiased Scene Graph Genera...

Head and Tail Localization of C. elegans

C. elegans is commonly used in neuroscience for behaviour analysis becau...

Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling

Sequential user behavior modeling plays a crucial role in online user-or...

Please sign up or login with your details

Forgot password? Click here to reset