Autonomous deep learning: Continual learning approach for dynamic environments

Andri Ashfahani, Mahardhika Pratama

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

63 Citations (Scopus)

Abstract

The feasibility of deep neural networks (DNNs) to address data stream problems still requires intensive study because of the static and offline nature of conventional deep learning approaches. A deep continual learning algorithm, namely autonomous deep learning (ADL), is proposed in this paper. Unlike traditional deep learning methods, ADL features a flexible structure where its network structure can be constructed from scratch with the absence of initial network structure via the self-constructing network structure. ADL specifically addresses catastrophic forgetting by having a different-depth structure which is capable of achieving a trade-off between plasticity and stability. Network significance (NS) formula is proposed to drive the hidden nodes growing and pruning mechanism. Drift detection scenario (DDS) is put forward to signal distributional changes in data streams which induce the creation of a new hidden layer. Maximum information compression index (MICI) method plays an important role as a complexity reduction module eliminating redundant layers. The efficacy of ADL is numerically validated under the prequential test-then-train procedure in lifelong environments using nine popular data stream problems. The numerical results demonstrate that ADL consistently outperforms recent continual learning methods while characterizing the automatic construction of network structures.

Original languageEnglish
Title of host publicationSIAM International Conference on Data Mining, SDM 2019
PublisherSociety for Industrial and Applied Mathematics Publications
Pages666-674
Number of pages9
ISBN (Electronic)9781611975673
DOIs
Publication statusPublished - 2019
Externally publishedYes
Event19th SIAM International Conference on Data Mining, SDM 2019 - Calgary, Canada
Duration: 2 May 20194 May 2019

Publication series

NameSIAM International Conference on Data Mining, SDM 2019

Conference

Conference19th SIAM International Conference on Data Mining, SDM 2019
Country/TerritoryCanada
CityCalgary
Period2/05/194/05/19

Fingerprint

Dive into the research topics of 'Autonomous deep learning: Continual learning approach for dynamic environments'. Together they form a unique fingerprint.

Cite this