Transverse-layer partitioning of artificial neural networks for image classification

Abstract

We discuss issues of modular learning in artificial neural networks and explore possibilities of the partial use of modules when the computational resources are limited. The proposed method is based on the ability of a wavelet transform to separate information into high-and low-frequency parts. Using the expertise gained in developing convolutional wavelet neural networks, the authors perform a transverse-layer partitioning of the network into modules for the further partial use on devices with low computational capability. The theoretical justification of this approach in the paper is supported by experimentally dividing the MNIST database into 2 and 4 modules before using them sequentially and measuring the respective accuracy and performance. When using the individual modules, a two-fold (or higher) performance gain is achieved. The theoretical statements are verified using an AlexNet-like network on the GTSRB dataset, with a performance gain of 33% per module with no loss of accuracy.