An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systems

by   Ahmed Haj Yahmed, et al.

Deep reinforcement learning (DRL) is increasingly applied in large-scale productions like Netflix and Facebook. As with most data-driven systems, DRL systems can exhibit undesirable behaviors due to environmental drifts, which often occur in constantly-changing production settings. Continual Learning (CL) is the inherent self-healing approach for adapting the DRL agent in response to the environment's conditions shifts. However, successive shifts of considerable magnitude may cause the production environment to drift from its original state. Recent studies have shown that these environmental drifts tend to drive CL into long, or even unsuccessful, healing cycles, which arise from inefficiencies such as catastrophic forgetting, warm-starting failure, and slow convergence. In this paper, we propose Dr. DRL, an effective self-healing approach for DRL systems that integrates a novel mechanism of intentional forgetting into vanilla CL to overcome its main issues. Dr. DRL deliberately erases the DRL system's minor behaviors to systematically prioritize the adaptation of the key problem-solving skills. Using well-established DRL algorithms, Dr. DRL is compared with vanilla CL on various drifted environments. Dr. DRL is able to reduce, on average, the healing time and fine-tuning episodes by, respectively, 18.74 helps agents to adapt to 19.63 vanilla CL while maintaining and even enhancing by up to 45 rewards for drifted environments that are resolved by both approaches.


page 1

page 6

page 9


Using Deep Reinforcement Learning to solve Optimal Power Flow problem with generator failures

Deep Reinforcement Learning (DRL) is being used in many domains. One of ...

Architecting and Visualizing Deep Reinforcement Learning Models

To meet the growing interest in Deep Reinforcement Learning (DRL), we so...

Multi-environment lifelong deep reinforcement learning for medical imaging

Deep reinforcement learning(DRL) is increasingly being explored in medic...

Testing of Deep Reinforcement Learning Agents with Surrogate Models

Deep Reinforcement Learning (DRL) has received a lot of attention from t...

Autonomous Exploration and Mapping for Mobile Robots via Cumulative Curriculum Reinforcement Learning

Deep reinforcement learning (DRL) has been widely applied in autonomous ...

Vehicle management in a modular production context using Deep Q-Learning

We investigate the feasibility of deploying Deep-Q based deep reinforcem...

The Ladder in Chaos: A Simple and Effective Improvement to General DRL Algorithms by Policy Path Trimming and Boosting

Knowing the learning dynamics of policy is significant to unveiling the ...

Please sign up or login with your details

Forgot password? Click here to reset