Quantifying the Reproducibility of Graph Neural Networks using Multigraph Brain Data

Graph neural networks (GNNs) have witnessed an unprecedented proliferation in tackling several problems in computer vision, computer-aided diagnosis, and related fields. While prior studies have focused on boosting the model accuracy, quantifying the reproducibility of the most discriminative features identified by GNNs is still an intact problem that yields concerns about their reliability in clinical applications in particular. Specifically, the reproducibility of biological markers across clinical datasets and distribution shifts across classes (e.g., healthy and disordered brains) is of paramount importance in revealing the underpinning mechanisms of diseases as well as propelling the development of personalized treatment. Motivated by these issues, we propose, for the first time, reproducibility-based GNN selection (RG-Select), a framework for GNN reproducibility assessment via the quantification of the most discriminative features (i.e., biomarkers) shared between different models. To ascertain the soundness of our framework, the reproducibility assessment embraces variations of different factors such as training strategies and data perturbations. Despite these challenges, our framework successfully yielded replicable conclusions across different training strategies and various clinical datasets. Our findings could thus pave the way for the development of biomarker trustworthiness and reliability assessment methods for computer-aided diagnosis and prognosis tasks. RG-Select code is available on GitHub at https://github.com/basiralab/RG-Select.


page 18

page 22


Investigating the Predictive Reproducibility of Federated Graph Neural Networks using Medical Datasets

Graph neural networks (GNNs) have achieved extraordinary enhancements in...

Incremental Training of Graph Neural Networks on Temporal Graphs under Distribution Shift

Current graph neural networks (GNNs) are promising, especially when the ...

Graph Neural Networks in Computer Vision – Architectures, Datasets and Common Approaches

Graph Neural Networks (GNNs) are a family of graph networks inspired by ...

GraphGallery: A Platform for Fast Benchmarking and Easy Development of Graph Neural Networks Based Intelligent Software

Graph Neural Networks (GNNs) have recently shown to be powerful tools fo...

Quantifying Reproducibility in NLP and ML

Reproducibility has become an intensely debated topic in NLP and ML over...

Volumetric parcellation of the right ventricle for regional geometric and functional assessment

In clinical practice, assessment of right ventricle (RV) is primarily do...

Please sign up or login with your details

Forgot password? Click here to reset