• Login
• Admin

Phan, Kien Tuong (2019) Parallel circuit - a modular neural network architecture. PhD thesis, University of Nottingham.

PDF (Thesis - as examined) - Repository staff only - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader Download (11MB) Request Thesis

Abstract

One of the obstacles that hinder the development of Artificial Neural Networks (ANNs) is the heavy computational cost of the training process. In an attempt to address this problem, I proposed a lightweight model named Parallel Circuits (PCs), with an emphasis on modularity. One of the key inspirations for the proposed model is the human retina, which consists of various cell types that only respond to particular visual stimuli. Similarly, conventional ANNs with high redundancy are decomposed into semi-independent modules, which is deemed to provide more efficient learning, both in terms of speed and generalizability.

Owing to the benefits of having fewer connections, the PC models were empirically shown to be considerably faster, especially when implemented in larger models. I also pursued the ability of automatic problem decomposition, and discovered that diversifying the learning process in each circuit strongly benefits the generalization of the proposed model. PC was shown to be advantageous in term of sparsity, which is highly correlated to modularity.

DropCircuit, a regularizer that targets circuits, was introduced to further enhance their specialities. Together with PCs, DropCircuit outperformed models with dense connectivity in several experiments. The circuit-level DropCircuit also exhibited better performance compared to conventional DropOut in conjunction with both PC and non-PC configurations, demonstrating the benefits of modularity. The modularity was further enhanced by imposing a set of biologically inspired constraints. Circuits are modelled as either excitatory or inhibitory types with contrastive properties. Modified PC networks were shown to discover sparse and part-based representations, showing further improvement in generalization.

Item Type:	Thesis (University of Nottingham only) (PhD)
Supervisors:	Maul, Tomas Henrique Vu, Tuong Thuy
Subjects:	Q Science > QA Mathematics > QA 75 Electronic computers. Computer science
Faculties/Schools:	University of Nottingham, Malaysia > Faculty of Science and Engineering — Science > School of Computer Science
Item ID:	56941
Depositing User:	PHAN, KIEN TUONG
Date Deposited:	29 Jul 2019 04:40
Last Modified:	07 May 2020 11:16
URI:	https://eprints.nottingham.ac.uk/id/eprint/56941

Actions (Archive Staff Only)

Edit View