Efficient Computation of Substring Equivalence Classes with Suffix Arrays

Kazuyuki Narisawa, Hideharu Hiratsuka, Shunsuke Inenaga, Hideo Bannai, Masayuki Takeda

研究成果: ジャーナルへの寄稿記事

1 引用 (Scopus)

抄録

This paper considers enumeration of substring equivalence classes introduced by Blumer et al. (J ACM 34(3):578–595, 1987). These equivalence classes were originally proposed to define a text indexing structure called compact directed acyclic word graphs (CDAWGs). These equivalence classes are also useful for text analysis, since they group together redundant substrings with essentially identical occurrences. In this paper, we present how to enumerate these equivalence classes using only suffix arrays and two auxiliary arrays (rank arrays and lcp arrays), in O(n) time for a given string of length n over the integer alphabet. The proposed method overcomes all the existing algorithms which require O(nlog σ) time, where σ is the alphabet size. Our experimental results show that the proposed method is also practically faster and more memory efficient than the existing ones. Furthermore, we propose an O(n)-time algorithm which constructs the CDAWG of an input string over the integer alphabet. This algorithm is based on the above-mentioned algorithm to enumerate equivalence classes. Our experiments show that the proposed method runs faster than the existing algorithm on large alphabets.

元の言語英語
ページ(範囲)291-318
ページ数28
ジャーナルAlgorithmica
79
発行部数2
DOI
出版物ステータス出版済み - 10 1 2017

Fingerprint

Suffix Array
Equivalence classes
Equivalence class
Strings
Text Indexing
Text Analysis
Integer
Graph in graph theory
Enumeration
Data storage equipment
Experimental Results
Experiment
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Computer Science Applications
  • Applied Mathematics

これを引用

Efficient Computation of Substring Equivalence Classes with Suffix Arrays. / Narisawa, Kazuyuki; Hiratsuka, Hideharu; Inenaga, Shunsuke; Bannai, Hideo; Takeda, Masayuki.

:: Algorithmica, 巻 79, 番号 2, 01.10.2017, p. 291-318.

研究成果: ジャーナルへの寄稿記事

@article{727592feb90846959097aa1bfbe94d57,
title = "Efficient Computation of Substring Equivalence Classes with Suffix Arrays",
abstract = "This paper considers enumeration of substring equivalence classes introduced by Blumer et al. (J ACM 34(3):578–595, 1987). These equivalence classes were originally proposed to define a text indexing structure called compact directed acyclic word graphs (CDAWGs). These equivalence classes are also useful for text analysis, since they group together redundant substrings with essentially identical occurrences. In this paper, we present how to enumerate these equivalence classes using only suffix arrays and two auxiliary arrays (rank arrays and lcp arrays), in O(n) time for a given string of length n over the integer alphabet. The proposed method overcomes all the existing algorithms which require O(nlog σ) time, where σ is the alphabet size. Our experimental results show that the proposed method is also practically faster and more memory efficient than the existing ones. Furthermore, we propose an O(n)-time algorithm which constructs the CDAWG of an input string over the integer alphabet. This algorithm is based on the above-mentioned algorithm to enumerate equivalence classes. Our experiments show that the proposed method runs faster than the existing algorithm on large alphabets.",
author = "Kazuyuki Narisawa and Hideharu Hiratsuka and Shunsuke Inenaga and Hideo Bannai and Masayuki Takeda",
year = "2017",
month = "10",
day = "1",
doi = "10.1007/s00453-016-0178-z",
language = "English",
volume = "79",
pages = "291--318",
journal = "Algorithmica",
issn = "0178-4617",
publisher = "Springer New York",
number = "2",

}

TY - JOUR

T1 - Efficient Computation of Substring Equivalence Classes with Suffix Arrays

AU - Narisawa, Kazuyuki

AU - Hiratsuka, Hideharu

AU - Inenaga, Shunsuke

AU - Bannai, Hideo

AU - Takeda, Masayuki

PY - 2017/10/1

Y1 - 2017/10/1

N2 - This paper considers enumeration of substring equivalence classes introduced by Blumer et al. (J ACM 34(3):578–595, 1987). These equivalence classes were originally proposed to define a text indexing structure called compact directed acyclic word graphs (CDAWGs). These equivalence classes are also useful for text analysis, since they group together redundant substrings with essentially identical occurrences. In this paper, we present how to enumerate these equivalence classes using only suffix arrays and two auxiliary arrays (rank arrays and lcp arrays), in O(n) time for a given string of length n over the integer alphabet. The proposed method overcomes all the existing algorithms which require O(nlog σ) time, where σ is the alphabet size. Our experimental results show that the proposed method is also practically faster and more memory efficient than the existing ones. Furthermore, we propose an O(n)-time algorithm which constructs the CDAWG of an input string over the integer alphabet. This algorithm is based on the above-mentioned algorithm to enumerate equivalence classes. Our experiments show that the proposed method runs faster than the existing algorithm on large alphabets.

AB - This paper considers enumeration of substring equivalence classes introduced by Blumer et al. (J ACM 34(3):578–595, 1987). These equivalence classes were originally proposed to define a text indexing structure called compact directed acyclic word graphs (CDAWGs). These equivalence classes are also useful for text analysis, since they group together redundant substrings with essentially identical occurrences. In this paper, we present how to enumerate these equivalence classes using only suffix arrays and two auxiliary arrays (rank arrays and lcp arrays), in O(n) time for a given string of length n over the integer alphabet. The proposed method overcomes all the existing algorithms which require O(nlog σ) time, where σ is the alphabet size. Our experimental results show that the proposed method is also practically faster and more memory efficient than the existing ones. Furthermore, we propose an O(n)-time algorithm which constructs the CDAWG of an input string over the integer alphabet. This algorithm is based on the above-mentioned algorithm to enumerate equivalence classes. Our experiments show that the proposed method runs faster than the existing algorithm on large alphabets.

UR - http://www.scopus.com/inward/record.url?scp=84982285459&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84982285459&partnerID=8YFLogxK

U2 - 10.1007/s00453-016-0178-z

DO - 10.1007/s00453-016-0178-z

M3 - Article

VL - 79

SP - 291

EP - 318

JO - Algorithmica

JF - Algorithmica

SN - 0178-4617

IS - 2

ER -