TY - GEN

T1 - Conservativeness and monotonicity for learning algorithms

AU - Takimoto, Eiji

AU - Maruoka, Akira

PY - 1993

Y1 - 1993

N2 - In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.

AB - In the framework of PAC-learning model, relationships between learning processes and information compressing processes are investigated. Information compressing processes are formulated as weak Occam algorithms. A weak Occam algorithm is a deterministic polynomial time algorithm that, when given m examples of unknown function, outputs, with high probability, a representation of a function that is consistent with the examples and belongs to a function class with complexity o(m). It has been shown that a weak Occam algorithm is also a consistent PAC-learning algorithm. In this extended abstract, it is shown that the converse does not hold by giving a PAC-learning algorithm that is not a weak Occam algorithm, and also some natural properties, called conservativeness and monotonicity, for learning algorithms that might help the converse hold are given. In particular, the conditions that make a conservative PAC-learning algorithm a weak Occam algorithm are given, and it is shown that, under some natural conditions, a monotone PAC-learning algorithm for a hypothesis class can be transformed to a weak Occam algorithm without changing the hypothesis class.

UR - http://www.scopus.com/inward/record.url?scp=0027838951&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027838951&partnerID=8YFLogxK

U2 - 10.1145/168304.168381

DO - 10.1145/168304.168381

M3 - Conference contribution

AN - SCOPUS:0027838951

SN - 0897916115

SN - 9780897916110

T3 - Proc 6 Annu ACM Conf Comput Learn Theory

SP - 377

EP - 383

BT - Proc 6 Annu ACM Conf Comput Learn Theory

PB - Publ by ACM

T2 - Proceedings of the 6th Annual ACM Conference on Computational Learning Theory

Y2 - 25 July 1993 through 27 July 1993

ER -