Chair: Gary Kuhn, Siemens (USA)
K. M. Jarrin, MITRE Corporation (USA)
The Inverse Receptive Field Partition (IRFP) algorithm developed from the Receptive Field Partition (RFP) algorithm, operates on a wavelet basis function network. This technique is a fast method of basis function selection. It begins with the highest resolution wavelet basis functions as its seed functions. Like RFP, IRFP then uses receptive field activation principle during training. This principle insures only wavelets within a specified interval are chosen as candidates. By way of the receptive activation principle, wavelets are selected with the best fit from a precalculated development pool and moved to the main pool. The selection of wavelet coefficients for the main pool is based on best fit across all resolutions. Overall functional fit can be controlled by global MSE and pruning thresholds.
Clifford Sze-Tsan Choy, Hong Kong Polytechnic University (HONG KONG)
Wan-Chi Siu, Hong Kong Polytechnic University (HONG KONG)
Self-organizing model has been successfully applied to solve some combinatorial optimization problems, including travelling salesman problem, routing problem and cell- placement problem, but there has not much work reported on its application to solve graph partitioning problem. In this paper, we propose a novel mapping which has not been proposed before, with some changes to the original Kohonen's algorithm so as to enable it to solve a partitioning problem -- the bipartite subgraph problem. This new approach is compared with the maximum neural network for solving the same problem, and shows that the performance of our new approach is superior over the maximum neural network.
Lane Owsley, University of Washington (USA)
Les Atlas, University of Washington (USA)
Gary Bernard, Boeing Commercial Airline Group (USA)
Automatic feature extraction is a need in many current applications, including the monitoring of industrial tools. Currently available approaches suffer from a number of shortcomings. The Kohonen self-organizing neural network (SONN) has the potential to act as a feature extractor, but we find it benefits from several modifications. The purpose of these modifications is to cause feature variations to be aligned with the SONN indices so that the indices themselves can be used as measures of the features. The modified SONN is applied to the dull tool monitoring problem, and it is shown that the new algorithm extracts and characterizes useful features of the data.
Fu-Chiang Tsui, University of Pittsburgh (USA)
Mingui Sun, University of Pittsburgh (USA)
Ching-Chung Li, University of Pittsburgh (USA)
Robert J. Sclabassi, University of Pittsburgh (USA)
A new approach is presented for time-series modeling and prediction using recurrent neural networks(RNNs) and a discrete wavelet transform(DWT). A specific DWT, based on the cubic spline wavelet, produces a set of wavelet coefficients from coarse to fine scale levels. The RNN has its current output fed back to its input nodes, forming a nonlinear autoregressive model for predicting future wavelet coefficients. A predicted trend signal is obtained by constructing the interpolation function from the predicted wavelet coefficients at the coarsest scale level, V_0. This method has been applied to intracranial pressure data collected from head trauma patients in the intensive care unit. The method has been shown to be more efficient than one which uses raw data to train the RNN.
Chuan Wang, University of Florida (USA)
Jyh-Ming Kuo, University of Florida (USA)
Jose C. Principe, University of Florida (USA)
Traditionally, adaptive learning systems are classified into two distinct paradigms---supervised and unsupervised learning. Although a lot of results have been published in these two learning paradigms, the relations between them have been seldom investigated. In this paper we focus on the relationship between the two kinds of learning and show that in a linear network the supervised learning with mean square error(MSE) criterion is equivalent to the basic anti-Hebbian learning rule when the desired signal is a zero mean random noise independent of the input. At least for this case there is a simple relationship between the two apparent different learning paradigms.
Weibo Liang, University of Texas at Arlington (USA)
Michael T. Manry, University of Texas at Arlington (USA)
Qiang Yu, University of Texas at Arlington (USA)
Michael S. Dawson, University of Texas at Arlington (USA)
Adrian K. Fung, University of Texas at Arlington (USA)
In minimum mean square estimation, an estimate of a random parameter vector is obtained from a noisy input vector. Recently, it has been shown that the training error for neural network estimators is minimized when the neural network approximates the minimum mean square estimator. In this paper, we develop bounds on the variances of estimation error for the case where the input signal vector and the parameter vector are non-Gaussian. In neural network applications, the bounds represent target values for the network mean-square training error. First, we linearly transform the input and parameter vectors. The resulting transformed vectors are approximately Gaussian because of the central limit theorem, so stochastic Cramer-Rao bounds on the variances of their estimation errors are tight. Lastly, approximate bounds on variances of the original parameters' estimation errors are obtained.
Ibrahim M. Elfadel, Masimo Corporation (USA)
A left (resp. right) principal singular subspace of dimension p is the subspace spanned by the p left (resp. right) singular vectors corresponding to the p largest singular values of the cross-correlation matrix of two stochastic processes. In this paper, we study the global dynamics of a system of nonlinear ordinary differential equations (ODE's) that govern the unsupervised Hebbian learning of left and right principal singular subspaces from samples of the two stochastic processes. In particular, we show that these equations admit a simple Lyapunov function when they are restricted to a well defined smooth, compact manifold, and that they are related to a Matrix Riccati differential equation. Moreover, we show that in the case p = 1, the solutions of these ODE's can be given in closed form.
Qi Li, University of Rhode Island (USA)
Donald W. Tufts, University of Rhode Island (USA)
Investigations of the design of a Discriminant Neural Network (DNN) [1-3] have shown the advantages of sequential design of hidden nodes and pruning of the training data for improved classification and fast training time. The performance can be further improved by adding the capability of a nonlinear, principal component discriminant node. This type of hidden node is useful for separating classes which have common mean vectors and are overlapped on one other.
Helge B. D. Sorensen, Technical University of Denmark
Uwe Hartmann, Aalborg University (DENMARK)
Preben Hunnerup, Aalborg University (DENMARK)
This paper presents a new training algorithm for Self-structuring Hidden Control Neural (SHC) models, which we presented at a previous ICASSP. The SHC models were trained non-discriminatively for speech recognition applications. Better recognition performance can generally be achieved, if discriminative training is applied instead. Thus we developed a discriminative training algorithm for SHC models, where each SHC model for a specific speech pattern is trained with utterances of the pattern to be recognized and with other utterances. The discriminative training of SHC neural models has been tested on the TIDIGITS database.
Neil J. Bershad, University of California Irvine
Nicolas Cubaud, Universite de Toulouse (FRANCE)
John J. Shynk, University of California at Santa Barbara (USA)
The statistical learning behavior of the single-layer backpropagation algorithm was recently analyzed using a system identification formulation for noise- free training data [1,2]. Transient and steady-state results were obtained for the mean weight behavior, mean-square error (MSE), and probability of correct classification. This paper extends these results to the case of noisy training data. Three new analytical results are obtained: 1) the mean weights converge to finite values even when the bias terms are zero, 2) the MSE is bounded away from zero, and 3) the probability of correct classification does not converge to unity. However, over a wide range of signal-to-noise ratios (SNRs), the noisy training data does not have a significant effect on the perceptron stationary points relative to the weight fluctuations. Hence, one concludes that noisy training data has a relatively small effect on the ability of the perceptron to learn the model weight vector F.