StarCraft II: A New Challenge for Reinforcement Learning

by   Oriol Vinyals, et al.

This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a more difficult class of problems than considered in most prior work. It is a multi-agent problem with multiple players interacting; there is imperfect information due to a partially observed map; it has a large action space involving the selection and control of hundreds of units; it has a large state space that must be observed solely from raw input feature planes; and it has delayed credit assignment requiring long-term strategies over thousands of steps. We describe the observation, action, and reward specification for the StarCraft II domain and provide an open source Python-based interface for communicating with the game engine. In addition to the main game maps, we provide a suite of mini-games focusing on different elements of StarCraft II gameplay. For the main game maps, we also provide an accompanying dataset of game replay data from human expert players. We give initial baseline results for neural networks trained from this data to predict game outcomes and player actions. Finally, we present initial baseline results for canonical deep reinforcement learning agents applied to the StarCraft II domain. On the mini-games, these agents learn to achieve a level of play that is comparable to a novice player. However, when trained on the main game, these agents are unable to make significant progress. Thus, SC2LE offers a new and challenging environment for exploring deep reinforcement learning algorithms and architectures.


page 3

page 5

page 7


TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

Starcraft II (SCII) is widely considered as the most challenging Real Ti...

Deep RL Agent for a Real-Time Action Strategy Game

We introduce a reinforcement learning environment based on Heroic - Magi...

Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning

The development of deep reinforcement learning (DRL) has benefited from ...

Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning

Analysis of invasive sports such as soccer is challenging because the ga...

An Introduction of mini-AlphaStar

StarCraft II (SC2) is a real-time strategy game, in which players produc...

Evaluating Soccer Player: from Live Camera to Deep Reinforcement Learning

Scientifically evaluating soccer players represents a challenging Machin...

DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

Games are abstractions of the real world, where artificial agents learn ...

Please sign up or login with your details

Forgot password? Click here to reset