Particle Swarm Based Hyper-Parameter Optimization for Machine Learned Interatomic Potentials

by   Suresh Kondati Natarajan, et al.

Modeling non-empirical and highly flexible interatomic potential energy surfaces (PES) using machine learning (ML) approaches is becoming popular in molecular and materials research. Training an ML-PES is typically performed in two stages: feature extraction and structure-property relationship modeling. The feature extraction stage transforms atomic positions into a symmetry-invariant mathematical representation. This representation can be fine-tuned by adjusting on a set of so-called "hyper-parameters" (HPs). Subsequently, an ML algorithm such as neural networks or Gaussian process regression (GPR) is used to model the structure-PES relationship based on another set of HPs. Choosing optimal values for the two sets of HPs is critical to ensure the high quality of the resulting ML-PES model. In this paper, we explore HP optimization strategies tailored for ML-PES generation using a custom-coded parallel particle swarm optimizer (available freely at We employ the smooth overlap of atomic positions (SOAP) descriptor in combination with GPR-based Gaussian approximation potentials (GAP) and optimize HPs for four distinct systems: a toy C dimer, amorphous carbon, α-Fe, and small organic molecules (QM9 dataset). We propose a two-step optimization strategy in which the HPs related to the feature extraction stage are optimized first, followed by the optimization of the HPs in the training stage. This strategy is computationally more efficient than optimizing all HPs at the same time by means of significantly reducing the number of ML models needed to be trained to obtain the optimal HPs. This approach can be trivially extended to other combinations of descriptor and ML algorithm and brings us another step closer to fully automated ML-PES generation.


Symbolic Regression in Materials Science: Discovering Interatomic Potentials from Data

Particle-based modeling of materials at atomic scale plays an important ...

Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials

Machine learning techniques allow a direct mapping of atomic positions a...

Molecular-orbital-based Machine Learning for Open-shell and Multi-reference Systems with Kernel Addition Gaussian Process Regression

We introduce a novel machine learning strategy, kernel addition Gaussian...

Graphical Gaussian Process Regression Model for Aqueous Solvation Free Energy Prediction of Organic Molecules in Redox Flow Battery

The solvation free energy of organic molecules is a critical parameter i...

Localized Coulomb Descriptors for the Gaussian Approximation Potential

We introduce a novel class of localized atomic environment representatio...

DScribe: Library of Descriptors for Machine Learning in Materials Science

DScribe is a software package for machine learning that provides popular...

Atomistic structure search using local surrogate mode

We describe a local surrogate model for use in conjunction with global s...

Please sign up or login with your details

Forgot password? Click here to reset