Autonomous deep learning: Incremental learning of deep neural networks for evolving data streams

It was recently demonstrated that network evolution strategies can be used to handle evolving data streams. However, most of these strategies are applied on shallow network structure making it unable to benefit from the power of depth. The application of deep neural networks (DNNs) for evolving data streams is hindered by its static nature which cannot adapt to rapidly changing environments. In this paper, we demonstrate that recently proposed DNN approaches, Autonomous Deep Learning (ADL) and Deep Evolving Denoising Autoencoder (DEVDAN), are able to address evolving data stream problems by adopting a flexible network structure DNN. In other words, these approaches manage to incrementally construct its network structure from scratch in respect to the variation of data. Furthermore, these methods work with the absence of a user-defined threshold and fully operate in the single-pass learning fashion. The multiple hidden nodes growing and adaptive synapses pruning are proposed to boost the predictive performance. When tested on popular datasets, the modified version of ADL and DEVDAN outperformed its original version and existing strategies while maintaining the space and computational complexity to an acceptable level. Also, data streams related research directions to deal with semi-supervised learning are discussed.