CONSTRUCTION OF NEURAL NETWORKS THAT DO NOT HAVE CRITICAL POINTS BASED ON HIERARCHICAL STRUCTURE
Journal Title: International Journal of Advanced Computer Science & Applications - Year 2013, Vol 4, Issue 9
Abstract
A critical point is a point at which the derivatives of an error function are all zero. It has been shown in the literature that critical points caused by the hierarchical structure of a real-valued neural network (NN) can be local minima or saddle points, although most critical points caused by the hierarchical structure are saddle points in the case of complex-valued neural networks. Several studies have demonstrated that singularity of those kinds has a negative effect on learning dynamics in neural networks. As described in this paper, the decomposition of high-dimensional neural networks into low-dimensional neural networks equivalent to the original neural networks yields neural networks that have no critical point based on the hierarchical structure. Concretely, the following three cases are shown: (a) A 2-2-2 real-valued NN is constructed from a 1-1-1 complex-valued NN. (b) A 4-4-4 real-valued NN is constructed from a 1-1-1 quaternionic NN. (c) A 2-2-2 complex-valued NN is constructed from a 1-1-1 quaternionic NN. Those NNs described above do not suffer from a negative effect by singular points during learning comparatively because they have no critical point based on a hierarchical structure.
Authors and Affiliations
Tohru Nitta
Keywords
Related Articles
- EP ID EP156967
- DOI 10.14569/IJACSA.2013.040911
- Views 97
- Downloads 0
A Novel Approach for Boosting Base Station Anonymity in a WSN
Nodes in a wireless sensor network scrutinize the nearby region and transmit their findings to the base station (BS) using multi-hop transmission. As the BS plays an important role in a wireless sensor network, therefore...
An Efficient Machine Learning Technique to Classify and Recognize Handwritten and Printed Digits of Sudoku Puzzle
In this paper, we propose a convolutional neural network model to recognize and classify handwritten and printed digits present in Sudoku puzzle, which is captured using smartphone camera from various magazines, and prin...
A Semantics for Concurrent Logic Programming Languages Based on Multiple-Valued Logic
In order to obtain an understanding of parallel logic thought it is necessary to establish a fully abstract model of the denotational semantics of logic programming languages. In this paper, a fixed point semantics for t...
The Method of Computer-Aided Design of a Bread Composition with Regard to Biomedical Requirements
A method for efficient software implementation of bread optimized multicomponent mixtures has been developed. These polycomposite mixtures have a chemical composition that meets the modern physiological standards of nutr...
A Conflict Resolution Strategy Selection Method (ConfRSSM) in Multi-Agent Systems
Selecting a suitable conflict resolution strategy when conflicts appear in multi-agent environments is a hard problem. There is a need to formulate a model for strategic decision making in selecting a strategy to resolve...