Constructing High Precision Knowledge Bases with Subjective and Factual Attributes

by   Ari Kobren, et al.

Knowledge bases (KBs) are the backbone of many ubiquitous applications and are thus required to exhibit high precision. However, for KBs that store subjective attributes of entities, e.g., whether a movie is "kid friendly", simply estimating precision is complicated by the inherent ambiguity in measuring subjective phenomena. In this work, we develop a method for constructing KBs with tunable precision--i.e., KBs that can be made to operate at a specific false positive rate, despite storing both difficult-to-evaluate subjective attributes and more traditional factual attributes. The key to our approach is probabilistically modeling user consensus with respect to each entity-attribute pair, rather than modeling each pair as either True or False. Uncertainty in the model is explicitly represented and used to control the KB's precision. We propose three neural networks for fitting the consensus model and evaluate each one on data from Google Maps--a large KB of locations and their subjective and factual attributes. The results demonstrate that our learned models are well-calibrated and thus can successfully be used to control the KB's precision. Moreover, when constrained to maintain 95 consensus model matches the F-score of a baseline that models each entity-attribute pair as a binary variable and does not support tunable precision. When unconstrained, our model dominates the same baseline by 12 F-score. Finally, we perform an empirical analysis of attribute-attribute correlations and show that leveraging them effectively contributes to reduced uncertainty and better performance in attribute prediction.


page 5

page 7


Language-Independent Sentiment Analysis Using Subjectivity and Positional Information

We describe a novel language-independent approach to the task of determi...

Multiplicative Coevolution Regression Models for Longitudinal Networks and Nodal Attributes

We introduce a simple and extendable coevolution model for the analysis ...

RAKA:Co-training of Relationships and Attributes for Cross-lingual Knowledge Alignment

Cross-lingual knowledge alignment suffers from the attribute heterogenei...

Attribute Acquisition in Ontology based on Representation Learning of Hierarchical Classes and Attributes

Attribute acquisition for classes is a key step in ontology construction...

Heterogeneous Entity Matching with Complex Attribute Associations using BERT and Neural Networks

Across various domains, data from different sources such as Baidu Baike ...

Consistency and Accuracy of CelebA Attribute Values

We report the first analysis of the experimental foundations of facial a...

Please sign up or login with your details

Forgot password? Click here to reset