Ray RLLib: A Composable and Scalable Reinforcement Learning Library

by   Eric Liang, et al.

Reinforcement learning (RL) algorithms involve the deep nesting of distinct components, where each component typically exhibits opportunities for distributed computation. Current RL libraries offer parallelism at the level of the entire program, coupling all the components together and making existing implementations difficult to extend, combine, and reuse. We argue for building composable RL components by encapsulating parallelism and resource requirements within individual components, which can be achieved by building on top of a flexible task-based programming model. We demonstrate this principle by building Ray RLLib on top of Ray and show that we can implement a wide range of state-of-the-art algorithms by composing and reusing a handful of standard components. This composability does not come at the cost of performance --- in our experiments, RLLib matches or exceeds the performance of highly optimized reference implementations. Ray RLLib is available as part of Ray at https://github.com/ray-project/ray/.


page 1

page 2

page 3

page 4


MushroomRL: Simplifying Reinforcement Learning Research

MushroomRL is an open-source Python library developed to simplify the pr...

Ray Interference: a Source of Plateaus in Deep Reinforcement Learning

Rather than proposing a new method, this paper investigates an issue pre...

Ray: A Distributed Framework for Emerging AI Applications

The next generation of AI applications will continuously interact with t...

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

We present Tianshou, a highly modularized python library for deep reinfo...

Faster Math Functions, Soundly

Standard library implementations of functions like sin and exp optimize ...

RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning

Reinforcement learning (RL) tasks are challenging to implement, execute ...

ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectives

We present ShinRL, an open-source library specialized for the evaluation...

Please sign up or login with your details

Forgot password? Click here to reset