Generalized Regression Neural Networks and the Implementation with Matlab
Generalized Regression Neural Networks (GRNN) is a special case of Radial Basis Networks (RBN). Compared with its competitor, e.g. standard feedforward neural network, GRNN has several advantages. First of all, the structure of a GRNN is relatively simple and static with 2 layers, namely pattern and summation layers. Once the input goes through each unit in the pattern layer, the relationship between the input and the response would be “memorized” and stored in the unit. As a result, # of units in the pattern layer is equal to # of observations in the training sample. In each pattern unit, a Gaussian PDF would be applied to the network input such that
Theta = EXP[-0.5 * (X – u) `(X – u) / ( Sigma ^2)]
where Theta is the output from pattern units, X is the input, u is training vector stored in the unit, and Sigma is a positive constant known as “spread” or “smooth parameter”. Once Theta is computed, it is passed to the summation layer to calculate Y|X = SUM(Y * Theta) / SUM(Theta), where Y|X is the prediction conditional on X and Y is the response in the training sample. In addition to the above, other benefits of GRNN claimed by Specht (1991) include:
1) The network is able to learning from the training data by “1-pass” training in a fraction of the time it takes to train standard feedforward networks.
2) The spread, Sigma, is the only free parameter in the network, which often can be identified by the V-fold or Split-Sample cross validation.
3) Unlike standard feedforward networks, GRNN estimation is always able to converge to a global solution and won’t be trapped by a local minimum.
With respect to the implementation of GRNN, Matlab might be considered the best computing engine from my limited experience in terms of ease to use and fast speed. A demo is given below on how to use matlab to develop a GRNN and to identify an optimal value of Sigma using split-sample cross validation.
load credit Y = transpose(data(:, 2)); [n, m] = size(Y); train_index = 2:2:m; % SPLIT THE RESPONSE VECTOR INTO TRAINING AND TESTING train_Y = Y(train_index); test_Y = Y; test_Y(train_index) = ; % SPLIT X MATRIX INTO TRAINING AND TESTING X = transpose(data(:, 3:10)); train_X = X(:, train_index); test_X = X; test_X(:, train_index) = ; % STANDARDIZE X MATRIX IN TRAINING SET [train_X2, map] = mapstd(train_X); % STANDARDIZE X MATRIX IN TESTING SET test_X2 = mapstd('apply', test_X, map); % CHECK IF VARIANCE == 1 var(transpose(train_X2)) var(transpose(test_X2)) % TESTING DIFFERENT SPREAD OF RADIAL BASIS FUNCTION j = 0; for i = 1:0.02:2 % TRAIN A GRNN grnn = newgrnn(train_X2, train_Y, i); % CALCULATE THE PREDICTION FOR TESTING SET test_P = sim(grnn, test_X2); % COLLECT THE PERFORMANCE if j == 0 spread = i; perf = sse(test_Y - test_P); else spread = [spread, i]; perf = [perf, sse(test_Y - test_P)]; end; j = j + 1; end; plot(spread, perf, '-ro');
The plot below is generated by the matlab program. As shown, SSE reaches the minimal when Sigma is between 1.3 and 1.4, indicating a reasonable range of the optimal Spread value.