GraphCleaner: Detecting Mislabelled Samples in Popular Graph Learning Benchmarks

by   Yuwen Li, et al.

Label errors have been found to be prevalent in popular text, vision, and audio datasets, which heavily influence the safe development and evaluation of machine learning algorithms. Despite increasing efforts towards improving the quality of generic data types, such as images and texts, the problem of mislabel detection in graph data remains underexplored. To bridge the gap, we explore mislabelling issues in popular real-world graph datasets and propose GraphCleaner, a post-hoc method to detect and correct these mislabelled nodes in graph datasets. GraphCleaner combines the novel ideas of 1) Synthetic Mislabel Dataset Generation, which seeks to generate realistic mislabels; and 2) Neighborhood-Aware Mislabel Detection, where neighborhood dependency is exploited in both labels and base classifier predictions. Empirical evaluations on 6 datasets and 6 experimental settings demonstrate that GraphCleaner outperforms the closest baseline, with an average improvement of 0.14 in F1 score, and 0.16 in MCC. On real-data case studies, GraphCleaner detects real and previously unknown mislabels in popular graph benchmarks: PubMed, Cora, CiteSeer and OGB-arxiv; we find that at least 6.91 mislabelled or ambiguous, and simply removing these mislabelled data can boost evaluation performance from 86.71


page 1

page 2

page 3

page 4


Empirical study of Machine Learning Classifier Evaluation Metrics behavior in Massively Imbalanced and Noisy data

With growing credit card transaction volumes, the fraud percentages are ...

Classification of Infant Crying in Real-World Home Environments Using Deep Learning

In the domain of social signal processing, automated audio recognition i...

Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data

Numerous Bayesian Network (BN) structure learning algorithms have been p...

A Framework for Large Scale Synthetic Graph Dataset Generation

Recently there has been increasing interest in developing and deploying ...

Bridging the Gap: Enhancing the Utility of Synthetic Data via Post-Processing Techniques

Acquiring and annotating suitable datasets for training deep learning mo...

Blind Estimation of Audio Processing Graph

Musicians and audio engineers sculpt and transform their sounds by conne...

Adaptive Preferential Attached kNN Graph with Distribution-Awareness

Graph-based kNN algorithms have garnered widespread popularity for machi...

Please sign up or login with your details

Forgot password? Click here to reset