TY - JOUR
T1 - Forward-backward envelope for the sum of two nonconvex functions
T2 - Further properties and nonmonotone linesearch algorithms
AU - Themelis, Andreas
AU - Stella, Lorenzo
AU - Patrinos, Panagiotis
N1 - Funding Information:
∗Received by the editors June 16, 2016; accepted for publication (in revised form) April 10, 2018; published electronically August 21, 2018. http://www.siam.org/journals/siopt/28-3/M108024.html Funding: This work was supported by the Research Foundation Flanders (FWO) research projects G086518N and G086318N; KU Leuven internal funding StG/15/043; Fonds de la Recherche Scientifique – FNRS and the Fonds Wetenschappelijk Onderzoek – Vlaanderen under EOS Project 30468160 (SeLMA).
PY - 2018
Y1 - 2018
N2 - We propose ZeroFPR, a nonmonotone linesearch algorithm for minimizing the sumof two nonconvex functions, one of which is smooth and the other possibly nonsmooth. ZeroFPRis the first algorithm that, despite being fit for fully nonconvex problems and requiring only theblack-box oracle of forward-backward splitting (FBS)|namely evaluations of the gradient of thesmooth term and of the proximity operator of the nonsmooth one|achieves superlinear convergencerates under mild assumptions at the limit point when the linesearch directions satisfy a Dennis{Moŕecondition, and we show that this is the case for Broyden's quasi-Newton directions. Our approach isbased on the forward-backward envelope (FBE), an exact and strictly continuous penalty functionfor the original cost. Extending previous results we show that, despite being nonsmooth for fullynonconvex problems, the FBE still enjoys favorable first-and second-order properties which are keyfor the convergence results of ZeroFPR. Our theoretical results are backed up by promising numericalsimulations. On large-scale problems, by computing linesearch directions using limited-memoryquasi-Newton updates our algorithm greatly outperforms FBS and its accelerated variant (AFBS).
AB - We propose ZeroFPR, a nonmonotone linesearch algorithm for minimizing the sumof two nonconvex functions, one of which is smooth and the other possibly nonsmooth. ZeroFPRis the first algorithm that, despite being fit for fully nonconvex problems and requiring only theblack-box oracle of forward-backward splitting (FBS)|namely evaluations of the gradient of thesmooth term and of the proximity operator of the nonsmooth one|achieves superlinear convergencerates under mild assumptions at the limit point when the linesearch directions satisfy a Dennis{Moŕecondition, and we show that this is the case for Broyden's quasi-Newton directions. Our approach isbased on the forward-backward envelope (FBE), an exact and strictly continuous penalty functionfor the original cost. Extending previous results we show that, despite being nonsmooth for fullynonconvex problems, the FBE still enjoys favorable first-and second-order properties which are keyfor the convergence results of ZeroFPR. Our theoretical results are backed up by promising numericalsimulations. On large-scale problems, by computing linesearch directions using limited-memoryquasi-Newton updates our algorithm greatly outperforms FBS and its accelerated variant (AFBS).
UR - http://www.scopus.com/inward/record.url?scp=85055197407&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85055197407&partnerID=8YFLogxK
U2 - 10.1137/16M1080240
DO - 10.1137/16M1080240
M3 - Article
AN - SCOPUS:85055197407
VL - 28
SP - 2274
EP - 2303
JO - SIAM Journal on Optimization
JF - SIAM Journal on Optimization
SN - 1052-6234
IS - 3
ER -