Watch and learn – a generalized approach for transferrable learning in deep neural networks via physical principles

by   Kyle Sprague, et al.

Transfer learning refers to the use of knowledge gained while solving a machine learning task and applying it to the solution of a closely related problem. Such an approach has enabled scientific breakthroughs in computer vision and natural language processing where the weights learned in state-of-the-art models can be used to initialize models for other tasks which dramatically improve their performance and save computational time. Here we demonstrate an unsupervised learning approach augmented with basic physical principles that achieves fully transferrable learning for problems in statistical physics across different physical regimes. By coupling a sequence model based on a recurrent neural network to an extensive deep neural network, we are able to learn the equilibrium probability distributions and inter-particle interaction models of classical statistical mechanical systems. Our approach, distribution-consistent learning, DCL, is a general strategy that works for a variety of canonical statistical mechanical models (Ising and Potts) as well as disordered (spin-glass) interaction potentials. Using data collected from a single set of observation conditions, DCL successfully extrapolates across all temperatures, thermodynamic phases, and can be applied to different length-scales. This constitutes a fully transferrable physics-based learning in a generalizable approach.


page 1

page 2

page 3

page 4


Physics-based Neural Networks for Shape from Polarization

How should prior knowledge from physics inform a neural network solution...

Variational approach to unsupervised learning

Deep belief networks are used extensively for unsupervised stochastic le...

Deep Neural Networks with Short Circuits for Improved Gradient Learning

Deep neural networks have achieved great success both in computer vision...

A Neural Network Based Method with Transfer Learning for Genetic Data Analysis

Transfer learning has emerged as a powerful technique in many applicatio...

Deep Universal Graph Embedding Neural Network

Learning powerful data embeddings has become a center piece in machine l...

Sequence-based Machine Learning Models in Jet Physics

Sequence-based modeling broadly refers to algorithms that act on data th...

Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

Deep learning has achieved astonishing results on many tasks with large ...

Please sign up or login with your details

Forgot password? Click here to reset