A Bandit Framework for Optimal Selection of Reinforcement Learning Agents

by   Andreas Merentitis, et al.

Deep Reinforcement Learning has been shown to be very successful in complex games, e.g. Atari or Go. These games have clearly defined rules, and hence allow simulation. In many practical applications, however, interactions with the environment are costly and a good simulator of the environment is not available. Further, as environments differ by application, the optimal inductive bias (architecture, hyperparameters, etc.) of a reinforcement agent depends on the application. In this work, we propose a multi-arm bandit framework that selects from a set of different reinforcement learning agents to choose the one with the best inductive bias. To alleviate the problem of sparse rewards, the reinforcement learning agents are augmented with surrogate rewards. This helps the bandit framework to select the best agents early, since these rewards are smoother and less sparse than the environment reward. The bandit has the double objective of maximizing the reward while the agents are learning and selecting the best agent after a finite number of learning steps. Our experimental results on standard environments show that the proposed framework is able to consistently select the optimal agent after a finite number of steps, while collecting more cumulative reward compared to selecting a sub-optimal architecture or uniformly alternating between different agents.


page 1

page 2

page 3

page 4


Shaping Advice in Deep Reinforcement Learning

Reinforcement learning involves agents interacting with an environment t...

Experience enrichment based task independent reward model

For most reinforcement learning approaches, the learning is performed by...

Dealing with Sparse Rewards in Reinforcement Learning

Successfully navigating a complex environment to obtain a desired outcom...

Reinforcement Learning with Goal-Distance Gradient

Reinforcement learning usually uses the feedback rewards of environmenta...

PDRL: Multi-Agent based Reinforcement Learning for Predictive Monitoring

Reinforcement learning has been increasingly applied in monitoring appli...

A Comparison of Reward Functions in Q-Learning Applied to a Cart Position Problem

Growing advancements in reinforcement learning has led to advancements i...

Action Guidance: Getting the Best of Sparse Rewards and Shaped Rewards for Real-time Strategy Games

Training agents using Reinforcement Learning in games with sparse reward...

Please sign up or login with your details

Forgot password? Click here to reset