Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network

Abstract

The main problem of using standard optimization methods is the need to change all parameters in same-size steps, regardless of the behavior of the gradient. A more efficient way to optimize a neural network is to set adaptive step sizes for each parameter. Standard methods are based on the square roots of exponential estimates of the moments of the squares of past gradients and do not use the local variation in gradients. The paper presents methods of adaptive non-convex and be-lief-based optimization with a positive-negative estimate of the moments with the corresponding theoretical guarantees of convergence. These approaches allow the loss function to more accurately converge in the neighborhood of the global minimum in a smaller number of iterations. The utilization of transformed positive-negative moment estimates and an additional parameter that controls the step size allows one to avoid local extremes for achieving higher performance, compared to similar methods. The introduction of the developed algorithms into the learning process of various architectures of multimodal neural network systems for analyzing heterogeneous data has made it possible to increase the accuracy of recognizing pigmented skin lesions by 2.33 – 5.69 percentage points, compared to the original optimization methods. Multimodal neural network systems for analyzing heterogeneous dermatological data, using the proposed optimization algorithms, can be applied as a tool for auxiliary medical diagnostics, which will reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of pigmentary oncopathologies.