## Prototyping A General Regression Neural Network with SAS

Last time when I read the paper “A General Regression Neural Network” by Donald Specht, it was exactly 10 years ago when I was in the graduate school. After reading again this week, I decided to code it out with SAS macros and make this excellent idea available for the SAS community.

The prototype of GRNN consists of 2 SAS macros, %grnn_learn() for the training of a GRNN and %grnn_pred() for the prediction with a GRNN. The famous Boston Housing dataset is used to test these two macros with the result compared with the outcome from the R implementation below. In this exercise, it is assumed that the smoothing parameter **SIGMA** is known and equal to 0.55 in order to simplify the case.

pkgs <- c('MASS', 'doParallel', 'foreach', 'grnn') lapply(pkgs, require, character.only = T) registerDoParallel(cores = 8) data(Boston) X <- Boston[-14] st.X <- scale(X) Y <- Boston[14] boston <- data.frame(st.X, Y) pred_grnn <- function(x, nn){ xlst <- split(x, 1:nrow(x)) pred <- foreach(i = xlst, .combine = rbind) %dopar% { data.frame(pred = guess(nn, as.matrix(i)), i, row.names = NULL) } } grnn <- smooth(learn(boston, variable.column = ncol(boston)), sigma = 0.55) pred_grnn <- pred_grnn(boston[, -ncol(boston)], grnn) head(pred_grnn$pred, n = 10) # [1] 24.61559 23.22232 32.29610 32.57700 33.29552 26.73482 21.46017 20.96827 # [9] 16.55537 20.25247

The first SAS macro to train a GRNN is %grnn_learn() shown below. The purpose of this macro is store the whole specification of a GRNN in a SAS dataset after the simple 1-pass training with the development data. Please note that motivated by the idea of MongoDB, I use the key-value paired scheme to store the information of a GRNN.

libname data ''; data data.boston; infile 'housing.data'; input x1 - x13 y; run; %macro grnn_learn(data = , x = , y = , sigma = , nn_out = ); options mprint mlogic nocenter; ********************************************************; * THIS MACRO IS TO TRAIN A GENERAL REGRESSION NEURAL *; * NETWORK (SPECHT, 1991) AND STORE THE SPECIFICATION *; *------------------------------------------------------*; * INPUT PARAMETERS: *; * DATA : INPUT SAS DATASET *; * X : A LIST OF PREDICTORS IN THE NUMERIC FORMAT *; * Y : A RESPONSE VARIABLE IN THE NUMERIC FORMAT *; * SIGMA : THE SMOOTH PARAMETER FOR GRNN *; * NN_OUT: OUTPUT SAS DATASET CONTAINING THE GRNN *; * SPECIFICATION *; *------------------------------------------------------*; * AUTHOR: *; * WENSUI.LIU@53.COM *; ********************************************************; data _tmp1; set &data (keep = &x &y); where &y ~= .; array _x_ &x; _miss_ = 0; do _i_ = 1 to dim(_x_); if _x_[_i_] = . then _miss_ = 1; end; if _miss_ = 0 then output; run; proc summary data = _tmp1; output out = _avg_ (drop = _type_ _freq_) mean(&x) = ; run; proc summary data = _tmp1; output out = _std_ (drop = _type_ _freq_) std(&x) = ; run; proc standard data = _tmp1 mean = 0 std = 1 out = _data_; var &x; run; data &nn_out (keep = _neuron_ _key_ _value_); set _last_ end = eof; _neuron_ + 1; length _key_ $32; array _a_ &y &x; do _i_ = 1 to dim(_a_); if _i_ = 1 then _key_ = '_Y_'; else _key_ = upcase(vname(_a_[_i_])); _value_ = _a_[_i_]; output; end; if eof then do; _neuron_ = 0; _key_ = "_SIGMA_"; _value_ = σ output; set _avg_; array _b_ &x; do _i_ = 1 to dim(_b_); _neuron_ = -1; _key_ = upcase(vname(_b_[_i_])); _value_ = _b_[_i_]; output; end; set _std_; array _c_ &x; do _i_ = 1 to dim(_c_); _neuron_ = -2; _key_ = upcase(vname(_c_[_i_])); _value_ = _c_[_i_]; output; end; end; run; proc datasets library = work; delete _: / memtype = data; run; quit; ********************************************************; * END OF THE MACRO *; ********************************************************; %mend grnn_learn; %grnn_learn(data = data.boston, x = x1 - x13, y = y, sigma = 0.55, nn_out = data.grnn); proc print data = data.grnn (obs = 10) noobs; run; /* SAS PRINTOUT OF GRNN DATA: _neuron_ _key_ _value_ 1 _Y_ 24.0000 1 X1 -0.4194 1 X2 0.2845 1 X3 -1.2866 1 X4 -0.2723 1 X5 -0.1441 1 X6 0.4133 1 X7 -0.1199 1 X8 0.1401 1 X9 -0.9819 */

After the training of a GRNN, the macro %grnn_pred() would be used to generate predicted values from a test dataset with all predictors. As shown in the print-out, first 10 predicted values are identical to those generated with R.

libname data ''; %macro grnn_pred(data = , x = , id = NA, nn_in = , out = grnn_pred); options mprint mlogic nocenter; ********************************************************; * THIS MACRO IS TO GENERATE PREDICTED VALUES BASED ON *; * THE SPECIFICATION OF GRNN CREATED BY THE %GRNN_LEARN *; * MACRO *; *------------------------------------------------------*; * INPUT PARAMETERS: *; * DATA : INPUT SAS DATASET *; * X : A LIST OF PREDICTORS IN THE NUMERIC FORMAT *; * ID : AN ID VARIABLE (OPTIONAL) *; * NN_IN: INPUT SAS DATASET CONTAINING THE GRNN *; * SPECIFICATION GENERATED FROM %GRNN_LEARN *; * OUT : OUTPUT SAS DATASET WITH GRNN PREDICTIONS *; *------------------------------------------------------*; * AUTHOR: *; * WENSUI.LIU@53.COM *; ********************************************************; data _data_; set &data; array _x_ &x; _miss_ = 0; do _i_ = 1 to dim(_x_); if _x_[_i_] = . then _miss_ = 1; end; if _miss_ = 0 then output; run; data _data_; set _last_ (drop = _miss_); %if &id = NA %then %do; _id_ + 1; %end; %else %do; _id_ = &id; %end; run; proc sort data = _last_ sortsize = max nodupkey; by _id_; run; data _data_ (keep = _id_ _key_ _value_); set _last_; array _x_ &x; length _key_ $32; do _i_ = 1 to dim(_x_); _key_ = upcase(vname(_x_[_i_])); _value_ = _x_[_i_]; output; end; run; proc sql noprint; select _value_ ** 2 into :s2 from &nn_in where _neuron_ = 0; create table _last_ as select a._id_, a._key_, (a._value_ - b._value_) / c._value_ as _value_ from _last_ as a, &nn_in as b, &nn_in as c where compress(a._key_, ' ') = compress(b._key_, ' ') and compress(a._key_, ' ') = compress(c._key_, ' ') and b._neuron_ = -1 and c._neuron_ = -2; create table _last_ as select a._id_, b._neuron_, sum((a._value_ - b._value_) ** 2) as d2, mean(c._value_) as y, exp(-(calculated d2) / (2 * &s2)) as exp from _last_ as a, &nn_in as b, &nn_in as c where compress(a._key_, ' ') = compress(b._key_, ' ') and b._neuron_ = c._neuron_ and b._neuron_ > 0 and c._key_ = '_Y_' group by a._id_, b._neuron_; create table _last_ as select a._id_, sum(a.y * a.exp / b.sum_exp) as _pred_ from _last_ as a inner join (select _id_, sum(exp) as sum_exp from _last_ group by _id_) as b on a._id_ = b._id_ group by a._id_; quit; proc sort data = _last_ out = &out sortsize = max; by _id_; run; ********************************************************; * END OF THE MACRO *; ********************************************************; %mend grnn_pred; %grnn_pred(data = data.boston, x = x1 - x13, nn_in = data.grnn); proc print data = grnn_pred (obs = 10) noobs; run; /* SAS PRINTOUT: _id_ _pred_ 1 24.6156 2 23.2223 3 32.2961 4 32.5770 5 33.2955 6 26.7348 7 21.4602 8 20.9683 9 16.5554 10 20.2525 */

After the development of these two macros, I also compare predictive performances between GRNN and OLS regression. It turns out that GRNN consistently outperforms OLS regression even with a wide range of **SIGMA** values. With a reasonable choice of **SIGMA** value, even a GRNN developed with 10% of the whole Boston Housing dataset is able to generalize well and yield a R^2 > 0.8 based upon the rest 90% data.