Peer Learning for Unbiased Scene Graph Generation

by   Liguang Zhou, et al.

In this paper, we propose a novel framework dubbed peer learning to deal with the problem of biased scene graph generation (SGG). This framework uses predicate sampling and consensus voting (PSCV) to encourage different peers to learn from each other, improving model diversity and mitigating bias in SGG. To address the heavily long-tailed distribution of predicate classes, we propose to use predicate sampling to divide and conquer this issue. As a result, the model is less biased and makes more balanced predicate predictions. Specifically, one peer may not be sufficiently diverse to discriminate between different levels of predicate distributions. Therefore, we sample the data distribution based on frequency of predicates into sub-distributions, selecting head, body, and tail classes to combine and feed to different peers as complementary predicate knowledge during the training process. The complementary predicate knowledge of these peers is then ensembled utilizing a consensus voting strategy, which simulates a civilized voting process in our society that emphasizes the majority opinion and diminishes the minority opinion. This approach ensures that the learned representations of each peer are optimally adapted to the various data distributions. Extensive experiments on the Visual Genome dataset demonstrate that PSCV outperforms previous methods. We have established a new state-of-the-art (SOTA) on the SGCls task by achieving a mean of 31.6.


CAME: Context-aware Mixture-of-Experts for Unbiased Scene Graph Generation

The scene graph generation has gained tremendous progress in recent year...

Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation

An unbiased scene graph generation (SGG) algorithm referred to as Skew C...

Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generation

Despite the huge progress in scene graph generation in recent years, its...

Taking A Closer Look at Visual Relation: Unbiased Video Scene Graph Generation with Decoupled Label Learning

Current video-based scene graph generation (VidSGG) methods have been fo...

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Today, scene graph generation(SGG) task is largely limited in realistic ...

CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation

Scene graphs are semantic abstraction of images that encourage visual un...

Triple Correlations-Guided Label Supplementation for Unbiased Video Scene Graph Generation

Video-based scene graph generation (VidSGG) is an approach that aims to ...

Please sign up or login with your details

Forgot password? Click here to reset