Towards solving neural networks with optimization trajectory search

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Modern gradient based optimization methods for deep neural networks demonstrate outstanding results on image classification tasks. However, methods that do not rely on gradient feedback fail to tackle deep network optimization. In the field of evolutionary computation, applying evolutionary algorithms directly to network weights remains to be an unresolved challenge. In this paper we examine a new framework for the evolution of deep nets. Based on the empirical analysis, we propose the use of linear sub-spaces of problems to search for promising optimization trajectories in parameter space, opposed to weight evolution. We show that linear sub-spaces of loss functions are sufficiently well-behaved to allow trajectory evaluation. Furthermore, we introduce fitness measure to show that it is possible to correctly categorize trajectories according to their distance from the optimal path. As such, this work introduces an alternative approach to evolutionary optimization of deep networks.

Original languageEnglish
Title of host publicationGECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion
PublisherAssociation for Computing Machinery, Inc
Pages75-76
Number of pages2
ISBN (Electronic)9781450367486
DOIs
Publication statusPublished - Jul 13 2019
Event2019 Genetic and Evolutionary Computation Conference, GECCO 2019 - Prague, Czech Republic
Duration: Jul 13 2019Jul 17 2019

Publication series

NameGECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion

Conference

Conference2019 Genetic and Evolutionary Computation Conference, GECCO 2019
CountryCzech Republic
CityPrague
Period7/13/197/17/19

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Theoretical Computer Science
  • Software

Fingerprint Dive into the research topics of 'Towards solving neural networks with optimization trajectory search'. Together they form a unique fingerprint.

  • Cite this

    Parsenadze, L. T., Vargas, D. V., & Fujita, T. (2019). Towards solving neural networks with optimization trajectory search. In GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion (pp. 75-76). (GECCO 2019 Companion - Proceedings of the 2019 Genetic and Evolutionary Computation Conference Companion). Association for Computing Machinery, Inc. https://doi.org/10.1145/3319619.3326796