Convergence properties of numerical simulation for neural networks

Kiichi Urahama

Research output: Contribution to journalArticle

Abstract

This paper discusses the convergence of the numerical solution of the neural network, whose stability is such that any trajectory asymptotically approaches the equilibrium solution. It is shown first that when the connection among neurons is weak, the upper limit of the step width, for which the asynchronous iteration converges, is the same as in the synchronous iteration. It is pointed out that the asynchronous method is expected to achieve a faster convergence in the parallel computer. On the other hand, when the synaptic connection is symmetrical, the asynchronous iteration does not converge in general unless the step width is set smaller than that of the synchronous iteration. It is shown also that the stiffness of the Hopfield network to solve the combinational optimization problem is enhanced with the increase of the network scale. Since the computational complexity is expected to increase rapidly in the forward Euler method, the iterative method based on the backward Euler method is proposed. It is shown that the proposed method has a better convergence property. Finally, the foregoing theoretical results are verified qualitatively through simple examples.

Original languageEnglish
Pages (from-to)28-35
Number of pages8
JournalElectronics and Communications in Japan (Part III: Fundamental Electronic Science)
Volume75
Issue number5
DOIs
Publication statusPublished - 1992

Fingerprint

Iterative methods
Neurons
Computational complexity
Trajectories
Stiffness
Neural networks
Computer simulation

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Cite this

Convergence properties of numerical simulation for neural networks. / Urahama, Kiichi.

In: Electronics and Communications in Japan (Part III: Fundamental Electronic Science), Vol. 75, No. 5, 1992, p. 28-35.

Research output: Contribution to journalArticle

@article{8f63357abfae43008da2280d8d62adcf,
title = "Convergence properties of numerical simulation for neural networks",
abstract = "This paper discusses the convergence of the numerical solution of the neural network, whose stability is such that any trajectory asymptotically approaches the equilibrium solution. It is shown first that when the connection among neurons is weak, the upper limit of the step width, for which the asynchronous iteration converges, is the same as in the synchronous iteration. It is pointed out that the asynchronous method is expected to achieve a faster convergence in the parallel computer. On the other hand, when the synaptic connection is symmetrical, the asynchronous iteration does not converge in general unless the step width is set smaller than that of the synchronous iteration. It is shown also that the stiffness of the Hopfield network to solve the combinational optimization problem is enhanced with the increase of the network scale. Since the computational complexity is expected to increase rapidly in the forward Euler method, the iterative method based on the backward Euler method is proposed. It is shown that the proposed method has a better convergence property. Finally, the foregoing theoretical results are verified qualitatively through simple examples.",
author = "Kiichi Urahama",
year = "1992",
doi = "10.1002/ecjc.4430750503",
language = "English",
volume = "75",
pages = "28--35",
journal = "Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)",
issn = "1042-0967",
publisher = "John Wiley and Sons Inc.",
number = "5",

}

TY - JOUR

T1 - Convergence properties of numerical simulation for neural networks

AU - Urahama, Kiichi

PY - 1992

Y1 - 1992

N2 - This paper discusses the convergence of the numerical solution of the neural network, whose stability is such that any trajectory asymptotically approaches the equilibrium solution. It is shown first that when the connection among neurons is weak, the upper limit of the step width, for which the asynchronous iteration converges, is the same as in the synchronous iteration. It is pointed out that the asynchronous method is expected to achieve a faster convergence in the parallel computer. On the other hand, when the synaptic connection is symmetrical, the asynchronous iteration does not converge in general unless the step width is set smaller than that of the synchronous iteration. It is shown also that the stiffness of the Hopfield network to solve the combinational optimization problem is enhanced with the increase of the network scale. Since the computational complexity is expected to increase rapidly in the forward Euler method, the iterative method based on the backward Euler method is proposed. It is shown that the proposed method has a better convergence property. Finally, the foregoing theoretical results are verified qualitatively through simple examples.

AB - This paper discusses the convergence of the numerical solution of the neural network, whose stability is such that any trajectory asymptotically approaches the equilibrium solution. It is shown first that when the connection among neurons is weak, the upper limit of the step width, for which the asynchronous iteration converges, is the same as in the synchronous iteration. It is pointed out that the asynchronous method is expected to achieve a faster convergence in the parallel computer. On the other hand, when the synaptic connection is symmetrical, the asynchronous iteration does not converge in general unless the step width is set smaller than that of the synchronous iteration. It is shown also that the stiffness of the Hopfield network to solve the combinational optimization problem is enhanced with the increase of the network scale. Since the computational complexity is expected to increase rapidly in the forward Euler method, the iterative method based on the backward Euler method is proposed. It is shown that the proposed method has a better convergence property. Finally, the foregoing theoretical results are verified qualitatively through simple examples.

UR - http://www.scopus.com/inward/record.url?scp=0026857210&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0026857210&partnerID=8YFLogxK

U2 - 10.1002/ecjc.4430750503

DO - 10.1002/ecjc.4430750503

M3 - Article

AN - SCOPUS:0026857210

VL - 75

SP - 28

EP - 35

JO - Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)

JF - Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)

SN - 1042-0967

IS - 5

ER -