Surrogate Models for Enhancing the Efficiency of Neuroevolution in Reinforcement Learning

by   Jörg Stork, et al.
TH Köln
Vrije Universiteit Amsterdam

In the last years, reinforcement learning received a lot of attention. One method to solve reinforcement learning tasks is Neuroevolution, where neural networks are optimized by evolutionary algorithms. A disadvantage of Neuroevolution is that it can require numerous function evaluations, while not fully utilizing the available information from each fitness evaluation. This is especially problematic when fitness evaluations become expensive. To reduce the cost of fitness evaluations, surrogate models can be employed to partially replace the fitness function. The difficulty of surrogate modeling for Neuroevolution is the complex search space and how to compare different networks. To that end, recent studies showed that a kernel based approach, particular with phenotypic distance measures, works well. These kernels compare different networks via their behavior (phenotype) rather than their topology or encoding (genotype). In this work, we discuss the use of surrogate model-based Neuroevolution (SMB-NE) using a phenotypic distance for reinforcement learning. In detail, we investigate a) the potential of SMB-NE with respect to evaluation efficiency and b) how to select adequate input sets for the phenotypic distance measure in a reinforcement learning problem. The results indicate that we are able to considerably increase the evaluation efficiency using dynamic input sets.


Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance Kernels

In NeuroEvolution, the topologies of artificial neural networks are opti...

Distance-based Kernels for Surrogate Model-based Neuroevolution

The topology optimization of artificial neural networks can be particula...

Learning to Generate Genotypes with Neural Networks

Neural networks and evolutionary computation have a rich intertwined his...

Linear Combination of Distance Measures for Surrogate Models in Genetic Programming

Surrogate models are a well established approach to reduce the number of...

Towards co-evolution of fitness predictors and Deep Neural Networks

Deep neural networks proved to be a very useful and powerful tool with m...

A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning

The integration of Reinforcement Learning (RL) and Evolutionary Algorith...

Data-efficient Neuroevolution with Kernel-Based Surrogate Models

Surrogate-assistance approaches have long been used in computationally e...

Please sign up or login with your details

Forgot password? Click here to reset