Abstract

A new model selection algorithm is established to determine the best number of hidden neurons for radial basis function neural networks. We used a Bayesian information-based criterion to select the best number of hidden neurons. The new algorithm grows the number of hidden neurons while the Bayesian information-based criterion is used for improvement. The optimal parameter values of a current neural network are used in the subsequent architecture. The computational results are compared with the trial-and-error approach through publicly available data sets. It is found that the new algorithm is suitable to improve the performance of the neural networks automatically. The root mean square error function is used to measure out-of-sample performance.

Keywords:

Model selection, radial basis neural network, schwarz information criterion,

References

1. Akaike, H., 1974. A new look at the statistical model identification. IEEE T. Automat. Contr., 19(6): 716-723.
   CrossRef    
2. Bache, K. and M. Lichman, 2013. UCI machine learning repository. School of Information and Computer Science, University of California, Irvine, Retrieved from: http://archive.ics.uci.edu/ml.
   Direct Link
3. Bors, A.G. and M. Gabbouj, 1994. Minimal topology for a radial basis functions neural network for pattern classification. Digit. Signal Process, 4(3): 173-188.
   CrossRef    
4. Broomhead, D.S. and D. Lowe, 1988. Radial basis functions, multi-variable functional interpolation and adaptive networks. Technical Report, DTIC Document.
   
5. Burnham, K.P. and D.R. Anderson, 2002. Model Selection and Multimodel Inference: A Practical Information-theoretic Approach. 2nd Edn., Springer, New York.
   
6. Cong, R.G. and M. Brady, 2012. The interdependence between rainfall and temperature: Copula analyses. Sci. World J., 2012: 11.
   CrossRef    PMid:23213286 PMCid:PMC3504421    
7. Du Toit, J.V., 2006. Automated construction of generalized additive neural networks for predictive data mining. Ph.D. Thesis, North-West University, Potchefstroom Campus.
   
8. Ghodsi, A. and D. Schuurmans, 2003. Automatic basis selection techniques for RBF networks. Neural Networks, 16(5-6): 809-816.
   CrossRef    
9. Gow, J.A. and C.D. Manning, 1999. Development of a photovoltaic array model for use in power-electronics simulation studies. IEE P-Elect. Pow. Appl., 146(2): 193-200.
   
10. Hamadneh, N., S. Sathasivam, S.L. Tilahun and O.H. Choon, 2012. Learning logic programming in radial basis function network via genetic algorithm. J. Appl. Sci., 12(9): 840-847.
    CrossRef    
11. Heertjes, M. and R. Verstappen, 2014. Self-tuning in integral sliding mode control with a Levenberg–Marquardt algorithm. Mechatronics, 24(4): 385-393.
    CrossRef    
12. Kingston, G.B., H.R. Maier and M.F. Lambert, 2005. A bayesian approach to artificial neural network model selection. Proceeding of the International Congress on Modelling and Simulation (MODSIM, 05). Melbourne, Vic., pp: 1853-1859.
    
13. Lampariello, F. and M. Sciandrone, 2001. Efficient training of RBF neural networks for pattern recognition. IEEE T. Neural Networ., 12(5): 1235-1242.
    CrossRef    PMid:18249950    
14. Lee, T.Y., S.A. Chen, H.Y. Hung and Y.Y. Ou, 2011. Incorporating distant sequence features and radial basis function networks to identify ubiquitin conjugation sites. PloS One, 6(3): e17331.
    CrossRef    PMid:21408064 PMCid:PMC3052307    
15. Leonardis, A. and H. Bischof, 1998. An efficient MDL-based construction of RBF networks. Neural Networks, 11(5): 963-973.
    CrossRef    
16. Lowe, D., 1989. Adaptive radial basis function nonlinearities, and the problem of generalisation. Proceeding of the 1st IEE International Conference on Artificial Neural Networks. London, pp: 171-175.
    
17. Moody, J. and C.J. Darken, 1989. Fast learning in networks of locally-tuned processing units. Neural Comput., 1(2): 281-294.
    CrossRef    
18. Moré, J.J., 1978. The Levenberg-Marquardt Algorithm: Implementation and Theory. In: Watson, G.A. (Ed.), Numerical Analysis. Springer, Berlin, Heidelberg, 630: 105-116.
    CrossRef    
19. Peng, J.X., K. Li and D.S. Huang, 2006. A hybrid forward algorithm for RBF neural network construction. IEEE T. Neural Networ., 17(6): 1439-1451.
    CrossRef    PMid:17131659    
20. Peng, J.X., K. Li and G.W. Irwin, 2007. A novel continuous forward algorithm for RBF neural modelling. IEEE T. Automat. Contr., 52(1): 117-122.
    CrossRef    
21. Schwarz, G., 1978. Estimating the dimension of a model. Ann. Stat., 6(2): 461-464.
    CrossRef    
22. Setiono, R., 2001. Feedforward neural network construction using cross validation. Neural Comput., 13(12): 2865-2877.
    CrossRef    PMid:11705414    
23. Tilahun, N.H.S.L., S. Sathasivam and O.H. Choon, 2013. Prey-predator algorithm as a new optimization technique using in radial basis function neural networks. Res. J. Appl. Sci., 8(7): 383-387.
    
24. Wang, T., H. Gao and J. Qiu, 2016. A combined adaptive neural network and nonlinear model predictive control for multirate networked industrial process control. IEEE T. Neural Networ. Lear. Syst., 27(2): 416-425.
    CrossRef    PMid:25898246    
25. Wang, Z.O. and T. Zhu, 2000. An efficient learning algorithm for improving generalization performance of radial basis function neural networks. Neural Networks, 13(4-5): 545-553.
    CrossRef    
26. Ye, N., 2003. The Handbook of Data Mining. Lawrence Erlbaum Associates, Publishers, Mahwah, NJ.
    
27. Zhao, Z.Q., X.D. Wu, C.Y. Lu, H. Glotin and J. Gao, 2014. Optimizing widths with PSO for center selection of Gaussian radial basis function networks. Sci. China Inform. Sci., 57(5): 1-17.
    CrossRef    

Competing interests

The authors have no competing interests.

Open Access Policy

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Copyright

The authors have no competing interests.