Sparse Reward Processes

by   Christos Dimitrakakis, et al.

We introduce a class of learning problems where the agent is presented with a series of tasks. Intuitively, if there is relation among those tasks, then the information gained during execution of one task has value for the execution of another task. Consequently, the agent is intrinsically motivated to explore its environment beyond the degree necessary to solve the current task it has at hand. We develop a decision theoretic setting that generalises standard reinforcement learning tasks and captures this intuition. More precisely, we consider a multi-stage stochastic game between a learning agent and an opponent. We posit that the setting is a good model for the problem of life-long learning in uncertain environments, where while resources must be spent learning about currently important tasks, there is also the need to allocate effort towards learning about aspects of the world which are not relevant at the moment. This is due to the fact that unpredictable future events may lead to a change of priorities for the decision maker. Thus, in some sense, the model "explains" the necessity of curiosity. Apart from introducing the general formalism, the paper provides algorithms. These are evaluated experimentally in some exemplary domains. In addition, performance bounds are proven for some cases of this problem.


page 1

page 2

page 3

page 4


Decoupled Learning of Environment Characteristics for Safe Exploration

Reinforcement learning is a proven technique for an agent to learn a tas...

An Information-Theoretic Analysis of Bayesian Reinforcement Learning

Building on the framework introduced by Xu and Raginksy [1] for supervis...

Comparative Evaluation of Multi-Agent Deep Reinforcement Learning Algorithms

Multi-agent deep reinforcement learning (MARL) suffers from a lack of co...

Developing cooperative policies for multi-stage reinforcement learning tasks

Many hierarchical reinforcement learning algorithms utilise a series of ...

BlockPuzzle - A Challenge in Physical Reasoning and Generalization for Robot Learning

In this work we propose a novel task framework under which a variety of ...

Unifying task specification in reinforcement learning

Reinforcement learning tasks are typically specified as Markov decision ...

Learning Embeddings for Sequential Tasks Using Population of Agents

We present an information-theoretic framework to learn fixed-dimensional...

Please sign up or login with your details

Forgot password? Click here to reset