Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines

Koji Ueno, Toyotaro Suzumura, Naoya Maruyama, Katsuki Fujisawa, Satoshi Matsuoka

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

There are many large-scale graphs in real world such as Web graphs and social graphs. The interest in large-scale graph analysis is growing in recent years. Breadth-First Search (BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid BFS algorithm by Beamer (Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’13, IEEE Computer Society, Washington, 2013), we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and several optimization techniques such as vertex reordering and load balancing. Our performance evaluation on K-Computer shows that our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously known best techniques.

Original languageEnglish
Pages (from-to)22-35
Number of pages14
JournalData Science and Engineering
Volume2
Issue number1
DOIs
Publication statusPublished - Mar 1 2017

Fingerprint

Data storage equipment
Supercomputers
Resource allocation
Data structures
Processing

All Science Journal Classification (ASJC) codes

  • Computational Mechanics
  • Computer Science Applications

Cite this

Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines. / Ueno, Koji; Suzumura, Toyotaro; Maruyama, Naoya; Fujisawa, Katsuki; Matsuoka, Satoshi.

In: Data Science and Engineering, Vol. 2, No. 1, 01.03.2017, p. 22-35.

Research output: Contribution to journalArticle

Ueno, Koji ; Suzumura, Toyotaro ; Maruyama, Naoya ; Fujisawa, Katsuki ; Matsuoka, Satoshi. / Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines. In: Data Science and Engineering. 2017 ; Vol. 2, No. 1. pp. 22-35.
@article{61d1ab3098ea4dab8e16372d3d9d77c7,
title = "Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines",
abstract = "There are many large-scale graphs in real world such as Web graphs and social graphs. The interest in large-scale graph analysis is growing in recent years. Breadth-First Search (BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid BFS algorithm by Beamer (Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’13, IEEE Computer Society, Washington, 2013), we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and several optimization techniques such as vertex reordering and load balancing. Our performance evaluation on K-Computer shows that our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously known best techniques.",
author = "Koji Ueno and Toyotaro Suzumura and Naoya Maruyama and Katsuki Fujisawa and Satoshi Matsuoka",
year = "2017",
month = "3",
day = "1",
doi = "10.1007/s41019-016-0024-y",
language = "English",
volume = "2",
pages = "22--35",
journal = "Data Science and Engineering",
issn = "2364-1185",
publisher = "Springer Berlin",
number = "1",

}

TY - JOUR

T1 - Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines

AU - Ueno, Koji

AU - Suzumura, Toyotaro

AU - Maruyama, Naoya

AU - Fujisawa, Katsuki

AU - Matsuoka, Satoshi

PY - 2017/3/1

Y1 - 2017/3/1

N2 - There are many large-scale graphs in real world such as Web graphs and social graphs. The interest in large-scale graph analysis is growing in recent years. Breadth-First Search (BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid BFS algorithm by Beamer (Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’13, IEEE Computer Society, Washington, 2013), we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and several optimization techniques such as vertex reordering and load balancing. Our performance evaluation on K-Computer shows that our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously known best techniques.

AB - There are many large-scale graphs in real world such as Web graphs and social graphs. The interest in large-scale graph analysis is growing in recent years. Breadth-First Search (BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid BFS algorithm by Beamer (Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’13, IEEE Computer Society, Washington, 2013), we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and several optimization techniques such as vertex reordering and load balancing. Our performance evaluation on K-Computer shows that our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously known best techniques.

UR - http://www.scopus.com/inward/record.url?scp=85047816698&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047816698&partnerID=8YFLogxK

U2 - 10.1007/s41019-016-0024-y

DO - 10.1007/s41019-016-0024-y

M3 - Article

AN - SCOPUS:85047816698

VL - 2

SP - 22

EP - 35

JO - Data Science and Engineering

JF - Data Science and Engineering

SN - 2364-1185

IS - 1

ER -