The Effect of the type of training algorithm for multi-layer perceptron neural network on the accuracy of monthly forecast of precipitation over Iran, case study: ECMWF model

Document Type : Research

Author

Assistant Professor, Climate Research Institute, Atmospheric Science and Meteorological Research Center, Mashhad, Iran

Abstract

Due to increasing atmospheric disasters in the Iran, accurate monthly and seasonal forecasts of rainfall as well as temperature, can help decision makers to better plan for the future. Meanwhile, machine learning methods are widely used today in predicting temperature and precipitation. For this purpose, the outputs of climate models are processed with the help of observational data and machine learning methods and a more accurate forecast of temperature and precipitation (or other climatic variables) are provided. In the meantime, methods based on multilayer perceptron artificial neural networks are widely used.
In a multi-layer perceptron artificial neural network, the design of the network architecture is very important and this design can directly affect the ability of the neural network to solve the problem. In designing network architecture, questions such as the number of neurons in each layer, the number of layers, network activity functions in each layer, etc. must be answered. In some cases, there are methods to answer each of the above questions, but in most cases, a suitable architecture for the specific problem under study must be found by trial and error. One of the important steps in using machine learning methods (in general) and especially the use of perceptron artificial neural network method is the training stage. During the neural network training process, which actually leads to solving a mathematical optimization problem, the optimal network weights are calculated as its adjustable parameters.
Today, various types of artificial neural networks are used in various fields of atmospheric science and climatology for purposes such as classification, regression and prediction. But the fundamental question in the use of artificial neural networks is how they are designed and built. One of the important points in using artificial neural networks that should be considered by designers is choosing the right algorithm for network training. In this paper, six different methods are considered for training a multilayer perceptron neural network including: Bayesian Regularization algorithm, Levenberg-Marquatt algorithm, Conjugate Gradient with Powell/Beale Restarts, BFGS Quasi-Newton algorithm, Scaled Conjugate Gradient and Fletcher-Powell Conjugate Gradient methods for monthly forecasting of precipitation that are reviewed and compared. In mathematical optimization methods based on derivatives and gradient vectors, the second-order derivative of the objective function, called the Hessian matrix, and its inverse, play an essential role in the calculations. On the other hand, with increasing the number of variables, the size of the matrix increases and its inverse calculation is computationally time consuming. Therefore, in the improved optimization methods, it is tried to approximate the inverse matrix of the objective function with some tricks.
Because the ECMWF model has six different lead times, 72 different models can be proposed for 12 different months of the year. For this purpose, data for the period 1993 to 2010 were used as network training data and data for the period 2011 to 2016 for testing. To evaluate the performance of different neural networks, three indices of correlation coefficient, mean square error and Nash-Sutcliffe index were used. Results indicated that the Bayesian Regularization and Levenberg-Marquatt, Conjugate Gradient with Powell/Beale Restarts outperforms other training algorithms.

Keywords

Main Subjects


آزادی، م.، واشانی، س. و حجام، س.، 1391، پیش‌بینی احتمالاتی بارش با استفاده از پس‌پردازش برون‌داد یک سامانه همادی، م. فیزیک زمین و فضا، 38(3)، 203-216.
صالحوند، ا.، گندمکار، ا. و فتاحی، ا.، 1399، پیش آگاهی بلندمدت بارش با استفاده از سیستم شبکه عصبی مطالعه موردی: حوضه کارون بزرگ، م. تحقیقات کاربردی علوم جغرافیایی (علوم جغرافیایی)، 20(59)، 97-81.
رحیمی نسب، م. و عامریان، ی.، 1398، پیش‌بینی بارش ماهانه در منطقه ایران با استفاده از ترکیب شبکه‏های عصبی مصنوعی و فیلتر کالمن توسعه یافته، م. اطلاعات جغرافیایی، 28(110)، 77-90.
عندلیب، غ.، نورانی، و.، منیری فر، ح. و شرقی، ا.، 1398، توسعه روش ترکیبی موجک-شبکه عصبی مصنوعی و ماشین بردار پشتیبان برای مدل‌سازی چند ایستگاهه بارش-رواناب با استفاده از ابزارهای خوشه‌بندی و اطلاعات مشترک، م. رویکردهای نوین در مهندسی عمران، 3(2)، 49-62.
نجیب‌زاده، ن.، و قادری، ک. و احمدی، م. 1398، بهره گیری از روش‌های رگرسیون بردار پشتیبان و شبکه عصبی مصنوعی در مدل‌سازی بارش رواناب (مطالعه موردی: حوضه آبریز سد صفارود)، م. آبیاری و زهکشی ایران، 13(6)، 1709-1720.
شافعی زاده، م.، فتحیان، ح. و نیک بخت شهبازی، ع.، 1398، شبیه‌‌سازی پیوسته بارش-رواناب با استفاده از شبکه‏های عصبی مصنوعی برمبنای انتخاب متغیرهای مؤثر ورودی با الگوریتم اطلاعات متقابل جزئی (PMI)، م. تحقیقات منابع آب ایران، 15(2)، 144-161.
Pakdaman, M., Falamarzi, Y., Babaeian, I. and Javanshiri, Z., 2020a, Post-processing of the North American multi-model ensemble for monthly forecast of precipitation based on neural network models. Theoretical and Applied Climatology, 141(1), 405-417.
Pakdaman, M., Naghab, S. S., Khazanedari, L., Malbousi, S. and Falamarzi, Y., 2020b, Lightning prediction using an ensemble learning approach for northeast of Iran. Journal of Atmospheric and Solar-Terrestrial Physics, 209, 105417.
Pakdaman, M., Falamarzi, Y., Yazdi, H. S., Ahmadian, A., Salahshour, S. and Ferrara, F., 2020c, A kernel least mean square algorithm for fuzzy differential equations and its application in earth’s energy balance model and climate. Alexandria Engineering Journal, 59(4), 2803-2810.
Pakdaman, M., Habibi Nokhandan, M. and Falamarzi, Y., 2021, Revisiting albedo from a fuzzy perspective, Kybernetes, https://doi.org/10.1108/K-12-2020-0834.
Cybenko, G., 1989, Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4), 303-314.
Hagan, M. T. and Menhaj, M. B., 1994, Training feedforward networks with the Marquardt algorithm. IEEE transactions on Neural Networks, 5(6), 989-993.
Powell, M. J. D., 1977, Restart procedures for the conjugate gradient method. Mathematical programming, 12(1), 241-254.
Lee, J., Kim, C. G., Lee, J. E., Kim, N. W. and Kim, H., 2018, Application of artificial neural networks to rainfall forecasting in the Geum River basin, Korea. Water, 10(10), 1448.
Zabbah, I., Roshani, A. R. and Khafage, A., 2018, Prediction of monthly rainfall using artificial neural network mixture approach, Case Study: Torbat-e Heydariyeh, Journal of the Earth and Space Physics, 44(4), 115-126.
Ahmadi, M. A., 2011, Prediction of asphaltene precipitation using artificial neural network optimized by imperialist competitive algorithm. Journal of Petroleum Exploration and Production Technology, 1(2-4), 99-106.
Khazanedari, L., Malbosi, S., Samadi Neghab, S., Pakdaman, M. and Javanshiri, Z., 2020, Extreme Climate Events in Iran during 2018. Nivar, 44(108-109), 68-78.
Izadi, N., Karakani, E. G., Saadatabadi, A. R., Shamsipour, A., Fattahi, E. and Habibi, M., 2021, Evaluation of ERA5 Precipitation Accuracy Based on Various Time Scales over Iran during 2000–2018. Water, 13(18), 2538.
Taghizadeh, E., Ahmadi-Givi, F., Brocca, L. and Sharifi, E., 2021, Evaluation of satellite/reanalysis precipitation products over Iran. International Journal of Remote Sensing, 42(9), 3474-3497.
Foresee, F. D. and Hagan, M. T., 1997, Gauss-Newton approximation to Bayesian learning. In Proceedings of international conference on neural networks (ICNN'97) (Vol. 3, pp. 1930-1935). IEEE.
Bazaraa, M. S., Sherali, H. D. and Shetty, C. M., 2013, Nonlinear programming: theory and algorithms. John Wiley & Sons.
Fan, Y., Krasnopolsky, V., van den Dool, H., Wu, C. Y. and Gottschalck, J., 2021, Using Artificial Neural Networks to Improve CFS Week 3-4 Precipitation and 2-Meter Air Temperature Forecasts. Weather and Forecasting.
Nguyen, H. N., Nguyen, T. A., Ly, H. B., Tran, V. Q., Nguyen, L. K., Nguyen, M. V. and Ngo, C. T., 2021, Prediction of daily and monthly rainfall using a backpropagation neural network. Journal of Applied Science and Engineering, 24(3), 367-379.
Anochi, J. A. and de Campos Velho, H. F., 2020, Neural network for seasonal climate precipitation prediction on the Brazil. Ciência e Natura, 42, 15.
Liu, Y., Zhao, Q., Yao, W., Ma, X., Yao, Y. and Liu, L., 2019, Short-term rainfall forecast model based on the improved Bp–nn algorithm. Scientific reports, 9(1), 1-12.