Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone linesearch algorithms

Andreas Themelis, Lorenzo Stella, Panagiotis Patrinos

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)

Abstract

We propose ZeroFPR, a nonmonotone linesearch algorithm for minimizing the sumof two nonconvex functions, one of which is smooth and the other possibly nonsmooth. ZeroFPRis the first algorithm that, despite being fit for fully nonconvex problems and requiring only theblack-box oracle of forward-backward splitting (FBS)|namely evaluations of the gradient of thesmooth term and of the proximity operator of the nonsmooth one|achieves superlinear convergencerates under mild assumptions at the limit point when the linesearch directions satisfy a Dennis{Moŕecondition, and we show that this is the case for Broyden's quasi-Newton directions. Our approach isbased on the forward-backward envelope (FBE), an exact and strictly continuous penalty functionfor the original cost. Extending previous results we show that, despite being nonsmooth for fullynonconvex problems, the FBE still enjoys favorable first-and second-order properties which are keyfor the convergence results of ZeroFPR. Our theoretical results are backed up by promising numericalsimulations. On large-scale problems, by computing linesearch directions using limited-memoryquasi-Newton updates our algorithm greatly outperforms FBS and its accelerated variant (AFBS).

Original languageEnglish
Pages (from-to)2274-2303
Number of pages30
JournalSIAM Journal on Optimization
Volume28
Issue number3
DOIs
Publication statusPublished - 2018
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science

Fingerprint

Dive into the research topics of 'Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone linesearch algorithms'. Together they form a unique fingerprint.

Cite this