Automatic construction of multi-layer perceptron network from streaming examples

Mahardhika Pratama, Choiru Za'in, Andri Ashfahani, Yew Soon Ong, Weiping Ding

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

42 Citations (Scopus)

Abstract

Autonomous construction of deep neural network (DNNs) is desired for data streams because it potentially offers two advantages: proper model's capacity and quick reaction to drift and shift. While self-organizing mechanism of DNNs remains an open issue, this task is even more challenging to be developed for standard multi-layer DNNs than that using the different-depth structures, because addition of a new layer results in information loss of previously trained knowledge. A Neural Network with Dynamically Evolved Capacity (NADINE) is proposed in this paper. NADINE features a fully open structure where its network structure, depth and width, can be automatically evolved from scratch in the online manner and without the use of problem-specific thresholds. NADINE is structured under a standard MLP architecture and the catastrophic forgetting issue during the hidden layer addition phase is resolved using the proposal of soft-forgetting and adaptive memory methods. The advantage of NADINE, namely elastic structure and online learning trait, is numerically validated using nine data stream classification and regression problems where it demonstrates performance's improvement over prominent algorithms in all problems. In addition, it is capable of dealing with data stream regression and classification problems equally well.

Original languageEnglish
Title of host publicationCIKM 2019 - Proceedings of the 28th ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Pages1171-1180
Number of pages10
ISBN (Electronic)9781450369763
DOIs
Publication statusPublished - 3 Nov 2019
Externally publishedYes
Event28th ACM International Conference on Information and Knowledge Management, CIKM 2019 - Beijing, China
Duration: 3 Nov 20197 Nov 2019

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Conference

Conference28th ACM International Conference on Information and Knowledge Management, CIKM 2019
Country/TerritoryChina
CityBeijing
Period3/11/197/11/19

Keywords

  • Concept Drifts
  • Continual Learning
  • Data Streams
  • Deep Learning
  • Online Learning

Fingerprint

Dive into the research topics of 'Automatic construction of multi-layer perceptron network from streaming examples'. Together they form a unique fingerprint.

Cite this