A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

Norikazu Takahashi, Jiro Katayama, Masato Seki, Junnichi Takeuchi

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

Original languageEnglish
Pages (from-to)221-250
Number of pages30
JournalComputational Optimization and Applications
Volume71
Issue number1
DOIs
Publication statusPublished - Sep 1 2018

Fingerprint

Non-negative Matrix Factorization
Global Analysis
Factorization
Convergence Analysis
Global Convergence
Multiplicative
Update
Convergence Properties
Computational methods
Subsequence
Stationary point
Computational Methods
Divergence
Optimization Problem
Sufficient Conditions

All Science Journal Classification (ASJC) codes

  • Control and Optimization
  • Computational Mathematics
  • Applied Mathematics

Cite this

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization. / Takahashi, Norikazu; Katayama, Jiro; Seki, Masato; Takeuchi, Junnichi.

In: Computational Optimization and Applications, Vol. 71, No. 1, 01.09.2018, p. 221-250.

Research output: Contribution to journalArticle

@article{c71cbd9b3b764f07ab7b4fcc04137f05,
title = "A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization",
abstract = "Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and R{\'e}nyi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.",
author = "Norikazu Takahashi and Jiro Katayama and Masato Seki and Junnichi Takeuchi",
year = "2018",
month = "9",
day = "1",
doi = "10.1007/s10589-018-9997-y",
language = "English",
volume = "71",
pages = "221--250",
journal = "Computational Optimization and Applications",
issn = "0926-6003",
publisher = "Springer Netherlands",
number = "1",

}

TY - JOUR

T1 - A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

AU - Takahashi, Norikazu

AU - Katayama, Jiro

AU - Seki, Masato

AU - Takeuchi, Junnichi

PY - 2018/9/1

Y1 - 2018/9/1

N2 - Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

AB - Multiplicative update rules are a well-known computational method for nonnegative matrix factorization. Depending on the error measure between two matrices, various types of multiplicative update rules have been proposed so far. However, their convergence properties are not fully understood. This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergent subsequence is a stationary point of the optimization problem. Using this condition, it is proved that many of the existing multiplicative update rules have the global convergence property if they are modified slightly so that all variables take positive values. This paper also proposes new multiplicative update rules based on Kullback–Leibler, Gamma, and Rényi divergences. It is shown that these three rules have the global convergence property if the same modification as above is made.

UR - http://www.scopus.com/inward/record.url?scp=85044089143&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044089143&partnerID=8YFLogxK

U2 - 10.1007/s10589-018-9997-y

DO - 10.1007/s10589-018-9997-y

M3 - Article

AN - SCOPUS:85044089143

VL - 71

SP - 221

EP - 250

JO - Computational Optimization and Applications

JF - Computational Optimization and Applications

SN - 0926-6003

IS - 1

ER -