Knowledge Graph Quality Evaluation under Incomplete Information

by   Xiaodong Li, et al.

Utilities of knowledge graphs (KGs) depend on their qualities. A KG that is of poor quality not only has little applicability but also leads to some unexpected errors. Therefore, quality evaluation for KGs is crucial and indispensable. Existing methods design many quality dimensions and calculate metrics in the corresponding dimensions based on details (i.e., raw data and graph structures) of KGs for evaluation. However, there are two major issues. On one hand, they consider the details as public information, which exposes the raw data and graph structures. These details are strictly confidential because they involve commercial privacy or others in practice. On the other hand, the existing methods focus on how much knowledge KGs have rather than KGs' practicability. To address the above problems, we propose a knowledge graph quality evaluation framework under incomplete information (QEII). The quality evaluation problem is transformed into an adversarial game, and the relative quality is evaluated according to the winner and loser. Participants of the game are KGs, and the adversarial gameplay is to question and answer (Q A). In the QEII, we generate and train a question model and an answer model for each KG. The question model of a KG first asks a certain number of questions to the other KG. Then it evaluates the answers returned by the answer model of the other KG and outputs a percentage score. The relative quality is evaluated by the scores, which measures the ability to apply knowledge. Q A messages are the only information that KGs exchange, without exposing any raw data and graph structure. Experimental results on two pairs of KGs demonstrate that, comparing with baselines, the QEII realizes a reasonable quality evaluation from the perspective of third-party evaluators under incomplete information.


Message Passing for Complex Question Answering over Knowledge Graphs

Question answering over knowledge graphs (KGQA) has evolved from simple ...

Steps to Knowledge Graphs Quality Assessment

Knowledge Graphs (KGs) have been popularized during the last decade, for...

Knowledge Graph Curation: A Practical Framework

Knowledge Graphs (KGs) have shown to be very important for applications ...

Towards Knowledge Graphs Validation through Weighted Knowledge Sources

The performance of applications, such as personal assistants, search eng...

Graph-augmented Learning to Rank for Querying Large-scale Knowledge Graph

Knowledge graph question answering (i.e., KGQA) based on information ret...

Knowledge Questions from Knowledge Graphs

We address the novel problem of automatically generating quiz-style know...

Please sign up or login with your details

Forgot password? Click here to reset