• Login
• Admin

Adamu, Abdullahi Shuaibu (2016) An empirical study towards efficient learning in artificial neural networks by neuronal diversity. PhD thesis, University of Nottingham.

Preview

PDF (Thesis - as examined) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (8MB) | Preview

Abstract

Artificial Neural Networks (ANN) are biologically inspired algorithms, and it is natural that it continues to inspire research in artificial neural networks. From the recent breakthrough of deep learning to the wake-sleep training routine, all have a common source of drawing inspiration: biology. The transfer functions of artificial neural networks play the important role of forming decision boundaries necessary for learning. However, there has been relatively little research on transfer function optimization compared to other aspects of neural network optimization. In this work, neuronal diversity - a property found in biological neural networks- is explored as a potentially promising method of transfer function optimization.

This work shows how neural diversity can improve generalization in the context of literature from the bias-variance decomposition and meta-learning. It then demonstrates that neural diversity - represented in the form of transfer function diversity- can exhibit diverse and accurate computational strategies that can be used as ensembles with competitive results without supplementing it with other diversity maintenance schemes that tend to be computationally expensive.

This work also presents neural network meta-features described as problem signatures sampled from models with diverse transfer functions for problem characterization. This was shown to meet the criteria of basic properties desired for any meta-feature, i.e. consistency for a problem and discriminatory for different problems. Furthermore, these meta-features were also used to study the underlying computational strategies adopted by the neural network models, which lead to the discovery of the strong discriminatory property of the evolved transfer function.

The culmination of this study is the co-evolution of neurally diverse neurons with their weights and topology for efficient learning. It is shown to achieve significant generalization ability as demonstrated by its average MSE of 0.30 on 22 different benchmarks with minimal resources (i.e. two hidden units). Interestingly, these are the properties associated with neural diversity. Thus, showing the properties of efficiency and increased computational capacity could be replicated with transfer function diversity in artificial neural networks.

Item Type:	Thesis (University of Nottingham only) (PhD)
Supervisors:	Maul, Tomas Bargiela, Andrzej
Keywords:	neuroevolution, artificial neural networks, biologically-inspired algorithms, transfer function optimisation
Subjects:	Q Science > QA Mathematics > QA 75 Electronic computers. Computer science
Faculties/Schools:	University of Nottingham, Malaysia > Faculty of Science and Engineering — Science > School of Computer Science
Item ID:	33799
Depositing User:	ADAMU, ABDULLAHI SHU'AIBU
Date Deposited:	21 Oct 2016 09:18
Last Modified:	23 Jan 2018 09:13
URI:	https://eprints.nottingham.ac.uk/id/eprint/33799

Actions (Archive Staff Only)

Edit View