Abstract
The feasibility of deep neural networks (DNNs) to address data streamproblems still requires intensive study because of the static and offlinenature of conventional deep learning approaches. A deep continual learningalgorithm, namely autonomous deep learning (ADL), is proposed in this paper.Unlike traditional deep learning methods, ADL features a flexible structurewhere its network structure can be constructed from scratch with the absence ofan initial network structure via the self-constructing network structure. ADLspecifically addresses catastrophic forgetting by having a different-depthstructure which is capable of achieving a trade-off between plasticity andstability. Network significance (NS) formula is proposed to drive the hiddennodes growing and pruning mechanism. Drift detection scenario (DDS) is putforward to signal distributional changes in data streams which induce thecreation of a new hidden layer. The maximum information compression index(MICI) method plays an important role as a complexity reduction moduleeliminating redundant layers. The efficacy of ADL is numerically validatedunder the prequential test-then-train procedure in lifelong environments usingnine popular data stream problems. The numerical results demonstrate that ADLconsistently outperforms recent continual learning methods while characterizingthe automatic construction of network structures.