Conservativeness and monotonicity for learning algorithms

Eiji Takimoto, Akira Maruoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.

Original languageEnglish
Title of host publicationProc 6 Annu ACM Conf Comput Learn Theory
Editors Anon
PublisherPubl by ACM
Pages377-383
Number of pages7
ISBN (Print)0897916115
Publication statusPublished - 1993
Externally publishedYes
EventProceedings of the 6th Annual ACM Conference on Computational Learning Theory - Santa Cruz, CA, USA
Duration: Jul 26 1993Jul 28 1993

Other

OtherProceedings of the 6th Annual ACM Conference on Computational Learning Theory
CitySanta Cruz, CA, USA
Period7/26/937/28/93

Fingerprint

Learning algorithms
Polynomials

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Takimoto, E., & Maruoka, A. (1993). Conservativeness and monotonicity for learning algorithms. In Anon (Ed.), Proc 6 Annu ACM Conf Comput Learn Theory (pp. 377-383). Publ by ACM.

Conservativeness and monotonicity for learning algorithms. / Takimoto, Eiji; Maruoka, Akira.

Proc 6 Annu ACM Conf Comput Learn Theory. ed. / Anon. Publ by ACM, 1993. p. 377-383.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takimoto, E & Maruoka, A 1993, Conservativeness and monotonicity for learning algorithms. in Anon (ed.), Proc 6 Annu ACM Conf Comput Learn Theory. Publ by ACM, pp. 377-383, Proceedings of the 6th Annual ACM Conference on Computational Learning Theory, Santa Cruz, CA, USA, 7/26/93.
Takimoto E, Maruoka A. Conservativeness and monotonicity for learning algorithms. In Anon, editor, Proc 6 Annu ACM Conf Comput Learn Theory. Publ by ACM. 1993. p. 377-383
Takimoto, Eiji ; Maruoka, Akira. / Conservativeness and monotonicity for learning algorithms. Proc 6 Annu ACM Conf Comput Learn Theory. editor / Anon. Publ by ACM, 1993. pp. 377-383
@inproceedings{b008dfb014fc43df84b480694df5279d,
title = "Conservativeness and monotonicity for learning algorithms",
abstract = "In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.",
author = "Eiji Takimoto and Akira Maruoka",
year = "1993",
language = "English",
isbn = "0897916115",
pages = "377--383",
editor = "Anon",
booktitle = "Proc 6 Annu ACM Conf Comput Learn Theory",
publisher = "Publ by ACM",

}

TY - GEN

T1 - Conservativeness and monotonicity for learning algorithms

AU - Takimoto, Eiji

AU - Maruoka, Akira

PY - 1993

Y1 - 1993

N2 - In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.

AB - In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.

UR - http://www.scopus.com/inward/record.url?scp=0027838951&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027838951&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0897916115

SP - 377

EP - 383

BT - Proc 6 Annu ACM Conf Comput Learn Theory

A2 - Anon, null

PB - Publ by ACM

ER -