Semantic Graph-enhanced Visual Network for Zero-shot Learning

by   Yang Hu, et al.

Zero-shot learning uses semantic attributes to connect the search space of unseen objects. In recent years, although the deep convolutional network brings powerful visual modeling capabilities to the ZSL task, its visual features have severe pattern inertia and lack of representation of semantic relationships, which leads to severe bias and ambiguity. In response to this, we propose the Graph-based Visual-Semantic Entanglement Network to conduct graph modeling of visual features, which is mapped to semantic attributes by using a knowledge graph, it contains several novel designs: 1. it establishes a multi-path entangled network with the convolutional neural network (CNN) and the graph convolutional network (GCN), which input the visual features from CNN to GCN to model the implicit semantic relations, then GCN feedback the graph modeled information to CNN features; 2. it uses attribute word vectors as the target for the graph semantic modeling of GCN, which forms a self-consistent regression for graph modeling and supervise GCN to learn more personalized attribute relations; 3. it fuses and supplements the hierarchical visual-semantic features refined by graph modeling into visual embedding. By promoting the semantic linkage modeling of visual features, our method outperforms state-of-the-art approaches on multiple representative ZSL datasets: AwA2, CUB, and SUN.


page 13

page 14


Zero-Shot Learning Based on Knowledge Sharing

Zero-Shot Learning (ZSL) is an emerging research that aims to solve the ...

Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs

We consider the problem of zero-shot recognition: learning a visual clas...

Boosting Generative Zero-Shot Learning by Synthesizing Diverse Features with Attribute Augmentation

The recent advance in deep generative models outlines a promising perspe...

Visually Aligned Word Embeddings for Improving Zero-shot Learning

Zero-shot learning (ZSL) highly depends on a good semantic embedding to ...

Semantic Enhanced Knowledge Graph for Large-Scale Zero-Shot Learning

Zero-Shot Learning has been a highlighted research topic in both vision ...

Benchmark Tests of Convolutional Neural Network and Graph Convolutional Network on HorovodRunner Enabled Spark Clusters

The freedom of fast iterations of distributed deep learning tasks is cru...

Enterprise Analytics using Graph Database and Graph-based Deep Learning

In a business-to-business (B2B) customer relationship management (CRM) u...

Please sign up or login with your details

Forgot password? Click here to reset