The impact of OCR accuracy and feature transformation on automatic text classification

Mayo Murata; Lazaro S.P. Busagala; Wataru Oyama; Tetsushi Wakabayashi; Fumitaka Kimura

doi:10.1007/11669487_45

The impact of OCR accuracy and feature transformation on automatic text classification

Mayo Murata, Lazaro S.P. Busagala, Wataru Oyama, Tetsushi Wakabayashi, Fumitaka Kimura

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

Digitization process of various printed documents involves generating texts by an OCR system for different applications including full-text retrieval and document organizations. However, OCR-generated texts have errors as per present OCR technology. Moreover, previous studies have revealed that as OCR accuracy decreases the classification performance also decreases. The reason for this is the use of absolute word frequency as feature vector. Representing OCR texts using absolute word frequency has limitations such as dependency on text length and word recognition rate consequently lower classification performance due to higher within-class variances. We describe feature transformation techniques which do not have such limitations and present improved experimental results from all used classifiers.

Original language	English
Title of host publication	Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings
Pages	506-517
Number of pages	12
DOIs	https://doi.org/10.1007/11669487_45
Publication status	Published - Jul 7 2006
Externally published	Yes
Event	7th International Workshop on Document Analysis Systems, DAS 2006 - Nelson, New Zealand Duration: Feb 13 2006 → Feb 15 2006

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	3872 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	7th International Workshop on Document Analysis Systems, DAS 2006
Country/Territory	New Zealand
City	Nelson
Period	2/13/06 → 2/15/06

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/11669487_45

Cite this

Murata, M., Busagala, L. S. P., Oyama, W., Wakabayashi, T., & Kimura, F. (2006). The impact of OCR accuracy and feature transformation on automatic text classification. In Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings (pp. 506-517). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3872 LNCS). https://doi.org/10.1007/11669487_45

The impact of OCR accuracy and feature transformation on automatic text classification. / Murata, Mayo; Busagala, Lazaro S.P.; Oyama, Wataru et al.

Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings. 2006. p. 506-517 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3872 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Murata, M, Busagala, LSP, Oyama, W, Wakabayashi, T & Kimura, F 2006, The impact of OCR accuracy and feature transformation on automatic text classification. in Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3872 LNCS, pp. 506-517, 7th International Workshop on Document Analysis Systems, DAS 2006, Nelson, New Zealand, 2/13/06. https://doi.org/10.1007/11669487_45

Murata M, Busagala LSP, Oyama W, Wakabayashi T, Kimura F. The impact of OCR accuracy and feature transformation on automatic text classification. In Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings. 2006. p. 506-517. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/11669487_45

Murata, Mayo ; Busagala, Lazaro S.P. ; Oyama, Wataru et al. / The impact of OCR accuracy and feature transformation on automatic text classification. Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings. 2006. pp. 506-517 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{8dff69bfb76349f796c79a36daff1709,

title = "The impact of OCR accuracy and feature transformation on automatic text classification",

abstract = "Digitization process of various printed documents involves generating texts by an OCR system for different applications including full-text retrieval and document organizations. However, OCR-generated texts have errors as per present OCR technology. Moreover, previous studies have revealed that as OCR accuracy decreases the classification performance also decreases. The reason for this is the use of absolute word frequency as feature vector. Representing OCR texts using absolute word frequency has limitations such as dependency on text length and word recognition rate consequently lower classification performance due to higher within-class variances. We describe feature transformation techniques which do not have such limitations and present improved experimental results from all used classifiers.",

author = "Mayo Murata and Busagala, {Lazaro S.P.} and Wataru Oyama and Tetsushi Wakabayashi and Fumitaka Kimura",

year = "2006",

month = jul,

day = "7",

doi = "10.1007/11669487_45",

language = "English",

isbn = "3540321403",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "506--517",

booktitle = "Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings",

note = "7th International Workshop on Document Analysis Systems, DAS 2006 ; Conference date: 13-02-2006 Through 15-02-2006",

}

TY - GEN

T1 - The impact of OCR accuracy and feature transformation on automatic text classification

AU - Murata, Mayo

AU - Busagala, Lazaro S.P.

AU - Oyama, Wataru

AU - Wakabayashi, Tetsushi

AU - Kimura, Fumitaka

PY - 2006/7/7

Y1 - 2006/7/7

N2 - Digitization process of various printed documents involves generating texts by an OCR system for different applications including full-text retrieval and document organizations. However, OCR-generated texts have errors as per present OCR technology. Moreover, previous studies have revealed that as OCR accuracy decreases the classification performance also decreases. The reason for this is the use of absolute word frequency as feature vector. Representing OCR texts using absolute word frequency has limitations such as dependency on text length and word recognition rate consequently lower classification performance due to higher within-class variances. We describe feature transformation techniques which do not have such limitations and present improved experimental results from all used classifiers.

AB - Digitization process of various printed documents involves generating texts by an OCR system for different applications including full-text retrieval and document organizations. However, OCR-generated texts have errors as per present OCR technology. Moreover, previous studies have revealed that as OCR accuracy decreases the classification performance also decreases. The reason for this is the use of absolute word frequency as feature vector. Representing OCR texts using absolute word frequency has limitations such as dependency on text length and word recognition rate consequently lower classification performance due to higher within-class variances. We describe feature transformation techniques which do not have such limitations and present improved experimental results from all used classifiers.

UR - http://www.scopus.com/inward/record.url?scp=33745557026&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745557026&partnerID=8YFLogxK

U2 - 10.1007/11669487_45

DO - 10.1007/11669487_45

M3 - Conference contribution

AN - SCOPUS:33745557026

SN - 3540321403

SN - 9783540321408

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 506

EP - 517

BT - Document Analysis Systems VII - 7th International Workshop, DAS 2006, Proceedings

T2 - 7th International Workshop on Document Analysis Systems, DAS 2006

Y2 - 13 February 2006 through 15 February 2006

ER -

The impact of OCR accuracy and feature transformation on automatic text classification

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this