Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification


Low, Cheng Yaw and Park, Jaewoo and Teoh, Andrew Beng Jin (2020) Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification. IEEE Transactions on Cybernetics, 50 (12). pp. 5021-5034. ISSN 2168-2267, 2168-2275

[img] Text
97.pdf - Published Version
Restricted to Repository staff only

Download (3MB)


Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained from end to end by backpropagation (BP), each S-DNN layer, that is, a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature relearning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN relearn by perturbing the intra/interclass variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains-faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we reveal that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results show that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.

Item Type: Article
Uncontrolled Keywords: Face recognition
Subjects: T Technology > TA Engineering (General). Civil engineering (General) > TA1501-1820 Applied optics. Photonics
Divisions: Faculty of Information Science and Technology (FIST)
Depositing User: Ms Suzilawati Abu Samah
Date Deposited: 22 Oct 2021 01:58
Last Modified: 22 Oct 2021 01:58


Downloads per month over past year

View ItemEdit (login required)