New method based on determining error surface for designing three layer neural networks

Baiquan Lu, Kotaro Hirasawa, Junichi Murata, Jinlu Hu, Chun Zhin Jin

Research output: Contribution to journalConference article

1 Citation (Scopus)

Abstract

A method is proposed for designing three layer neural networks that assures global minimization of errors. The commonly used gradient-based learning algorithm suffers form the local minima problem, however, it can be solved if the error surface becomes convex. In the paper a number of possible network structures are provided together with their gradient-based learning algorithms. For a given set of training data, an appropriate network structure, i.e. the number of hidden nodes, the types of activation function, and the connections between them, is determined. All of the proposed structures give convex error surfaces and thus solve the local minima problem. The difference between them is in the level of locality and generalization ability. A numerical example is provided that supports the present approach.

Original languageEnglish
Pages (from-to)III-384 - III-389
JournalProceedings of the IEEE International Conference on Systems, Man and Cybernetics
Volume3
Publication statusPublished - Dec 1 1999
Event1999 IEEE International Conference on Systems, Man, and Cybernetics 'Human Communication and Cybernetics' - Tokyo, Jpn
Duration: Oct 12 1999Oct 15 1999

Fingerprint

learning
Neural networks
Learning algorithms
Chemical activation
method

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Hardware and Architecture

Cite this

New method based on determining error surface for designing three layer neural networks. / Lu, Baiquan; Hirasawa, Kotaro; Murata, Junichi; Hu, Jinlu; Jin, Chun Zhin.

In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vol. 3, 01.12.1999, p. III-384 - III-389.

Research output: Contribution to journalConference article

@article{43a21d1e5ff5494fa6087a093cfce92a,
title = "New method based on determining error surface for designing three layer neural networks",
abstract = "A method is proposed for designing three layer neural networks that assures global minimization of errors. The commonly used gradient-based learning algorithm suffers form the local minima problem, however, it can be solved if the error surface becomes convex. In the paper a number of possible network structures are provided together with their gradient-based learning algorithms. For a given set of training data, an appropriate network structure, i.e. the number of hidden nodes, the types of activation function, and the connections between them, is determined. All of the proposed structures give convex error surfaces and thus solve the local minima problem. The difference between them is in the level of locality and generalization ability. A numerical example is provided that supports the present approach.",
author = "Baiquan Lu and Kotaro Hirasawa and Junichi Murata and Jinlu Hu and Jin, {Chun Zhin}",
year = "1999",
month = "12",
day = "1",
language = "English",
volume = "3",
pages = "III--384 -- III--389",
journal = "Proceedings of the IEEE International Conference on Systems, Man and Cybernetics",
issn = "0884-3627",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - New method based on determining error surface for designing three layer neural networks

AU - Lu, Baiquan

AU - Hirasawa, Kotaro

AU - Murata, Junichi

AU - Hu, Jinlu

AU - Jin, Chun Zhin

PY - 1999/12/1

Y1 - 1999/12/1

N2 - A method is proposed for designing three layer neural networks that assures global minimization of errors. The commonly used gradient-based learning algorithm suffers form the local minima problem, however, it can be solved if the error surface becomes convex. In the paper a number of possible network structures are provided together with their gradient-based learning algorithms. For a given set of training data, an appropriate network structure, i.e. the number of hidden nodes, the types of activation function, and the connections between them, is determined. All of the proposed structures give convex error surfaces and thus solve the local minima problem. The difference between them is in the level of locality and generalization ability. A numerical example is provided that supports the present approach.

AB - A method is proposed for designing three layer neural networks that assures global minimization of errors. The commonly used gradient-based learning algorithm suffers form the local minima problem, however, it can be solved if the error surface becomes convex. In the paper a number of possible network structures are provided together with their gradient-based learning algorithms. For a given set of training data, an appropriate network structure, i.e. the number of hidden nodes, the types of activation function, and the connections between them, is determined. All of the proposed structures give convex error surfaces and thus solve the local minima problem. The difference between them is in the level of locality and generalization ability. A numerical example is provided that supports the present approach.

UR - http://www.scopus.com/inward/record.url?scp=0033325184&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033325184&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:0033325184

VL - 3

SP - III-384 - III-389

JO - Proceedings of the IEEE International Conference on Systems, Man and Cybernetics

JF - Proceedings of the IEEE International Conference on Systems, Man and Cybernetics

SN - 0884-3627

ER -