An extension of a minimax approach to multiple classification

Tomonari Kitahara, Shinji Mizuno, Kazuhide Nakata

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

When mean vectors and covariance matrices of two classes are available in a binary classification problem, Lanckriet et al. [6] propose a minimax approach for finding a linear classifier which minimizes the worst-case (maximum) misclassification probability. In this paper, we extend the minimax approach to a multiple classification problem, where the number m of classes could be more than two. Assume that mean vectors and covariance matrices of all the classes are available, but no further assumptions are made with respect to class-conditional distributions. Then we define a problem for finding linear classifiers which minimize the worst-case misclassification probability ᾱ. Unfortunately, no efficient algorithms for solving the problem are known. So we introduce the maximum pairwise misclassification probability β̄ instead of ᾱ. It is shown that β̄ is a lower bound of ᾱ and a good approximation of ᾱ when m or ᾱ are small. We define a problem for finding linear classifiers which minimize the probability β̄ and show some basic properties of the problem. Then the problem is transformed to a parametric Second Order Cone Programming problem (SOCP). We propose an algorithm for solving the problem by using nice properties of it. We conduct preliminary numerical experiments and confirm that classifiers computed by our method work very well to benchmark problems.

Original languageEnglish
Pages (from-to)123-136
Number of pages14
JournalJournal of the Operations Research Society of Japan
Volume50
Issue number2
DOIs
Publication statusPublished - Jun 2007
Externally publishedYes

Fingerprint

Classifier
Minimax
Misclassification
Covariance matrix
Conditional distribution
Approximation
Benchmark
Numerical experiment
Lower bounds
Programming

All Science Journal Classification (ASJC) codes

  • Decision Sciences(all)
  • Management Science and Operations Research

Cite this

An extension of a minimax approach to multiple classification. / Kitahara, Tomonari; Mizuno, Shinji; Nakata, Kazuhide.

In: Journal of the Operations Research Society of Japan, Vol. 50, No. 2, 06.2007, p. 123-136.

Research output: Contribution to journalArticle

@article{07460db7965a4687827f5feb034d8504,
title = "An extension of a minimax approach to multiple classification",
abstract = "When mean vectors and covariance matrices of two classes are available in a binary classification problem, Lanckriet et al. [6] propose a minimax approach for finding a linear classifier which minimizes the worst-case (maximum) misclassification probability. In this paper, we extend the minimax approach to a multiple classification problem, where the number m of classes could be more than two. Assume that mean vectors and covariance matrices of all the classes are available, but no further assumptions are made with respect to class-conditional distributions. Then we define a problem for finding linear classifiers which minimize the worst-case misclassification probability ᾱ. Unfortunately, no efficient algorithms for solving the problem are known. So we introduce the maximum pairwise misclassification probability β̄ instead of ᾱ. It is shown that β̄ is a lower bound of ᾱ and a good approximation of ᾱ when m or ᾱ are small. We define a problem for finding linear classifiers which minimize the probability β̄ and show some basic properties of the problem. Then the problem is transformed to a parametric Second Order Cone Programming problem (SOCP). We propose an algorithm for solving the problem by using nice properties of it. We conduct preliminary numerical experiments and confirm that classifiers computed by our method work very well to benchmark problems.",
author = "Tomonari Kitahara and Shinji Mizuno and Kazuhide Nakata",
year = "2007",
month = "6",
doi = "10.15807/jorsj.50.123",
language = "English",
volume = "50",
pages = "123--136",
journal = "Journal of the Operations Research Society of Japan",
issn = "0453-4514",
publisher = "Operations Research Society of Japan",
number = "2",

}

TY - JOUR

T1 - An extension of a minimax approach to multiple classification

AU - Kitahara, Tomonari

AU - Mizuno, Shinji

AU - Nakata, Kazuhide

PY - 2007/6

Y1 - 2007/6

N2 - When mean vectors and covariance matrices of two classes are available in a binary classification problem, Lanckriet et al. [6] propose a minimax approach for finding a linear classifier which minimizes the worst-case (maximum) misclassification probability. In this paper, we extend the minimax approach to a multiple classification problem, where the number m of classes could be more than two. Assume that mean vectors and covariance matrices of all the classes are available, but no further assumptions are made with respect to class-conditional distributions. Then we define a problem for finding linear classifiers which minimize the worst-case misclassification probability ᾱ. Unfortunately, no efficient algorithms for solving the problem are known. So we introduce the maximum pairwise misclassification probability β̄ instead of ᾱ. It is shown that β̄ is a lower bound of ᾱ and a good approximation of ᾱ when m or ᾱ are small. We define a problem for finding linear classifiers which minimize the probability β̄ and show some basic properties of the problem. Then the problem is transformed to a parametric Second Order Cone Programming problem (SOCP). We propose an algorithm for solving the problem by using nice properties of it. We conduct preliminary numerical experiments and confirm that classifiers computed by our method work very well to benchmark problems.

AB - When mean vectors and covariance matrices of two classes are available in a binary classification problem, Lanckriet et al. [6] propose a minimax approach for finding a linear classifier which minimizes the worst-case (maximum) misclassification probability. In this paper, we extend the minimax approach to a multiple classification problem, where the number m of classes could be more than two. Assume that mean vectors and covariance matrices of all the classes are available, but no further assumptions are made with respect to class-conditional distributions. Then we define a problem for finding linear classifiers which minimize the worst-case misclassification probability ᾱ. Unfortunately, no efficient algorithms for solving the problem are known. So we introduce the maximum pairwise misclassification probability β̄ instead of ᾱ. It is shown that β̄ is a lower bound of ᾱ and a good approximation of ᾱ when m or ᾱ are small. We define a problem for finding linear classifiers which minimize the probability β̄ and show some basic properties of the problem. Then the problem is transformed to a parametric Second Order Cone Programming problem (SOCP). We propose an algorithm for solving the problem by using nice properties of it. We conduct preliminary numerical experiments and confirm that classifiers computed by our method work very well to benchmark problems.

UR - http://www.scopus.com/inward/record.url?scp=34547806046&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34547806046&partnerID=8YFLogxK

U2 - 10.15807/jorsj.50.123

DO - 10.15807/jorsj.50.123

M3 - Article

AN - SCOPUS:34547806046

VL - 50

SP - 123

EP - 136

JO - Journal of the Operations Research Society of Japan

JF - Journal of the Operations Research Society of Japan

SN - 0453-4514

IS - 2

ER -