Towards solving neural networks with optimization trajectory search

研究成果: 著書/レポートタイプへの貢献会議での発言

抄録

Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the field of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.

元の言語英語
ホスト出版物のタイトルGECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion
出版者Association for Computing Machinery, Inc
ページ75-76
ページ数2
ISBN(電子版)9781450367486
DOI
出版物ステータス出版済み - 7 13 2019
イベント2019 Genetic and Evolutionary Computation Conference, GECCO 2019 - Prague, チェコ共和国
継続期間: 7 13 20197 17 2019

出版物シリーズ

名前GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion

会議

会議2019 Genetic and Evolutionary Computation Conference, GECCO 2019
チェコ共和国
Prague
期間7/13/197/17/19

Fingerprint

Trajectory Optimization
Trajectories
Subspace
Neural Networks
Trajectory
Gradient
Neural networks
Network Optimization
Evolutionary Optimization
Optimal Path
Image Classification
Empirical Analysis
Evolutionary Computation
Loss Function
Evolutionary algorithms
Fitness
Optimization Methods
Parameter Space
Evolutionary Algorithms
Image classification

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Theoretical Computer Science
  • Software

これを引用

Parsenadze, L. T., Vargas, D. V., & Fujita, T. (2019). Towards solving neural networks with optimization trajectory search. : GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion (pp. 75-76). (GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion). Association for Computing Machinery, Inc. https://doi.org/10.1145/3319619.3326796

Towards solving neural networks with optimization trajectory search. / Parsenadze, Lia T.; Vargas, Danilo Vasconcellos; Fujita, Toshiyuki.

GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2019. p. 75-76 (GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion).

研究成果: 著書/レポートタイプへの貢献会議での発言

Parsenadze, LT, Vargas, DV & Fujita, T 2019, Towards solving neural networks with optimization trajectory search. : GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion. GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion, Association for Computing Machinery, Inc, pp. 75-76, 2019 Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, チェコ共和国, 7/13/19. https://doi.org/10.1145/3319619.3326796
Parsenadze LT, Vargas DV, Fujita T. Towards solving neural networks with optimization trajectory search. : GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc. 2019. p. 75-76. (GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion). https://doi.org/10.1145/3319619.3326796
Parsenadze, Lia T. ; Vargas, Danilo Vasconcellos ; Fujita, Toshiyuki. / Towards solving neural networks with optimization trajectory search. GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2019. pp. 75-76 (GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion).
@inproceedings{19aa0993ee0d47278bccc1305dabb283,
title = "Towards solving neural networks with optimization trajectory search",
abstract = "Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the field of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.",
author = "Parsenadze, {Lia T.} and Vargas, {Danilo Vasconcellos} and Toshiyuki Fujita",
year = "2019",
month = "7",
day = "13",
doi = "10.1145/3319619.3326796",
language = "English",
series = "GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion",
publisher = "Association for Computing Machinery, Inc",
pages = "75--76",
booktitle = "GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion",

}

TY - GEN

T1 - Towards solving neural networks with optimization trajectory search

AU - Parsenadze, Lia T.

AU - Vargas, Danilo Vasconcellos

AU - Fujita, Toshiyuki

PY - 2019/7/13

Y1 - 2019/7/13

N2 - Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the field of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.

AB - Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the field of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.

UR - http://www.scopus.com/inward/record.url?scp=85070649335&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070649335&partnerID=8YFLogxK

U2 - 10.1145/3319619.3326796

DO - 10.1145/3319619.3326796

M3 - Conference contribution

AN - SCOPUS:85070649335

T3 - GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion

SP - 75

EP - 76

BT - GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion

PB - Association for Computing Machinery, Inc

ER -