HKUST Institutional Repository >
Computer Science and Engineering >
CSE Doctoral Theses >
Please use this identifier to cite or link to this item:
|Title: ||Constructive algorithms for structure learning in feedforward neural networks|
|Authors: ||Kwok, Tin-Yau|
|Issue Date: ||1996 |
|Abstract: ||In recent years, multi-layer feedforward neural networks have been popularly used for pattern classification, function approximation and regression problems. Methods using standard back-propagation learning algorithm perform gradient descent only in the weight space of a network with fixed topology. Recently, various researchers have investigated different approaches that alter the network topology as learning proceeds. In this thesis, I concentrate on constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea of constructive algorithms is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. There are hurdles that constructive algorithms have to overcome, including:
1. How to train the new hidden unit? 2. Whether the constructive algorithms can produce a neural network function that is as close to an arbitrary target function as desired?
3. How to control the complexity of the new hidden unit? To address these issues, I develop a number of objective functions for training new hidden units. The theoretical convergence properties of a number of constructive algorithms, in which hidden units are added one by one in a greedy manner, are also examined. Moreover, I study how the integration of Bayesian regularization and constructive algorithms can lead to improved network performance. The approach is promising in that the regularization parameters can be automatically controlled in a disciplined manner without requiring manual setting.|
|Description: ||Thesis (Ph.D.)--Hong Kong University of Science and Technology, 1996|
xiii, 164 leaves : ill. ; 30 cm
HKUST Call Number: Thesis COMP 1996 Kwok
|Appears in Collections:||CSE Doctoral Theses|
Files in This Item:
All items in this Repository are protected by copyright, with all rights reserved.