Deep Kernel Methods Learn Better: From Cards to Process Optimization

by   Mani Valleti, et al.

The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards data set and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The jupyter notebooks that encapsulate the complete analysis accompany the article.


page 6

page 13

page 18

page 21

page 23

page 25

page 27


The Effectiveness of Variational Autoencoders for Active Learning

The high cost of acquiring labels is one of the main challenges in deplo...

Variational Adversarial Active Learning

Active learning aims to develop label-efficient algorithms by sampling t...

Reproducible, incremental representation learning with Rosetta VAE

Variational autoencoders are among the most popular methods for distilli...

An Exploration of Learnt Representations of W Jets

I present a Variational Autoencoder (VAE) trained on collider physics da...

Hyperparameter-free deep active learning for regression problems via query synthesis

In the past decade, deep active learning (DAL) has heavily focused upon ...

Data-Efficient Learning via Minimizing Hyperspherical Energy

Deep learning on large-scale data is dominant nowadays. The unprecedente...

Please sign up or login with your details

Forgot password? Click here to reset