Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies

by   Melody Wolk, et al.

Advancements in reinforcement learning (RL) have inspired new directions in intelligent automation of network defense. However, many of these advancements have either outpaced their application to network security or have not considered the challenges associated with implementing them in the real-world. To understand these problems, this work evaluates several RL approaches implemented in the second edition of the CAGE Challenge, a public competition to build an autonomous network defender agent in a high-fidelity network simulator. Our approaches all build on the Proximal Policy Optimization (PPO) family of algorithms, and include hierarchical RL, action masking, custom training, and ensemble RL. We find that the ensemble RL technique performs strongest, outperforming our other models and taking second place in the competition. To understand applicability to real environments we evaluate each method's ability to generalize to unseen networks and against an unknown attack strategy. In unseen environments, all of our approaches perform worse, with degradation varied based on the type of environmental change. Against an unknown attacker strategy, we found that our models had reduced overall performance even though the new strategy was less efficient than the ones our models trained on. Together, these results highlight promising research directions for autonomous network defense in the real world.


Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games

While traditionally a labour intensive task, the testing of game content...

Unified Emulation-Simulation Training Environment for Autonomous Cyber Agents

Autonomous cyber agents may be developed by applying reinforcement and d...

CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems

Given the success of reinforcement learning (RL) in various domains, it ...

Enabling A Network AI Gym for Autonomous Cyber Agents

This work aims to enable autonomous agents for network cyber operations ...

Network Defense is Not a Game

Research seeks to apply Artificial Intelligence (AI) to scale and extend...

Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout

A grand goal in AI is to build a robot that can accurately navigate base...

Experience Filter: Using Past Experiences on Unseen Tasks or Environments

One of the bottlenecks of training autonomous vehicle (AV) agents is the...

Please sign up or login with your details

Forgot password? Click here to reset