Please use this identifier to cite or link to this item: https://dspace.ncfu.ru/handle/20.500.12258/23476
Title: A new approach to training neural networks using natural gradient descent with momentum based on Dirichlet distributions
Authors: Abdulkadirov, R. I.
Абдулкадиров, Р. И.
Lyakhov, P. A.
Ляхов, П. А.
Keywords: Dirichlet distributions;Natural gradient descent;Pattern recognition;Machine learning
Issue Date: 2023
Citation: Abdulkadirov, R.I., Lyakhov, P.A. A new approach to training neural networks using natural gradient descent with momentum based on Dirichlet distributions // Computer Optics. - 2023. - 47 (1), pp. 160-169. - DOI: 10.18287/2412-6179-CO-1147
Series/Report no.: Computer Optics
Abstract: In this paper, we propose a natural gradient descent algorithm with momentum based on Dirichlet distributions to speed up the training of neural networks. This approach takes into account not only the direction of the gradients, but also the convexity of the minimized function, which significantly accelerates the process of searching for the extremes. Calculations of natural gradients based on Dirichlet distributions are presented, with the proposed approach introduced into an error backpropagation scheme. The results of image recognition and time series forecasting during the experiments show that the proposed approach gives higher accuracy and does not require a large number of iterations to minimize loss functions compared to the methods of stochastic gradient descent, adaptive moment estimation and adaptive parameter-wise diagonal quasi-Newton method for nonconvex stochastic optimization.
URI: http://hdl.handle.net/20.500.12258/23476
Appears in Collections:Статьи, проиндексированные в SCOPUS, WOS

Files in This Item:
File Description SizeFormat 
scopusresults 2527 .pdf
  Restricted Access
1.3 MBAdobe PDFView/Open
WoS 1593 .pdf
  Restricted Access
107.48 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.