Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments

by   Andri Ashfahani, et al.

The feasibility of deep neural networks (DNNs) to address data stream problems still requires intensive study because of the static and offline nature of conventional deep learning approaches. A deep continual learning algorithm, namely autonomous deep learning (ADL), is proposed in this paper. Unlike traditional deep learning methods, ADL features a flexible structure where its network structure can be constructed from scratch with the absence of initial network structure via the self-constructing network structure. ADL specifically addresses catastrophic forgetting by having a different-depth structure which is capable of achieving a trade-off between plasticity and stability. Network significance (NS) formula is proposed to drive the hidden nodes growing and pruning mechanism. Drift detection scenario (DDS) is put forward to signal distributional changes in data streams which induce the creation of a new hidden layer. Maximum information compression index (MICI) method plays an important role as a complexity reduction module eliminating redundant layers. The efficacy of ADL is numerically validated under the prequential test-then-train procedure in lifelong environments using nine popular data stream problems. The numerical results demonstrate that ADL consistently outperforms recent continual learning methods while characterizing the automatic construction of network structures.


page 1

page 2

page 3

page 4


An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams

Existing fuzzy neural networks (FNNs) are mostly developed under a shall...

Automatic Construction of Multi-layer Perceptron Network from Streaming Examples

Autonomous construction of deep neural network (DNNs) is desired for dat...

Unsupervised Continual Learning via Self-Adaptive Deep Clustering Approach

Unsupervised continual learning remains a relatively uncharted territory...

Parsimonious Random Vector Functional Link Network for Data Streams

The theory of random vector functional link network (RVFLN) has provided...

Autonomous Deep Learning: Incremental Learning of Denoising Autoencoder for Evolving Data Streams

The generative learning phase of Autoencoder (AE) and its successor Deno...

Increasing Depth of Neural Networks for Life-long Learning

Increasing neural network depth is a well-known method for improving neu...

Learning to SLAM on the Fly in Unknown Environments: A Continual Learning Approach for Drones in Visually Ambiguous Scenes

Learning to safely navigate in unknown environments is an important task...

Please sign up or login with your details

Forgot password? Click here to reset