Extreme scale breadth-first search on supercomputers

Koji Ueno, Toyotaro Suzumura, Naoya Maruyama, Katsuki Fujisawa, Satoshi Matsuoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Breadth-First Search(BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid-BFS algorithm by Beamer[3], we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and optimization techniques such as vertex reordering and load balancing. Performance evaluation on the K shows our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously-known best techniques.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
EditorsRonay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1040-1047
Number of pages8
ISBN (Electronic)9781467390040
DOIs
Publication statusPublished - Jan 1 2016
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: Dec 5 2016Dec 8 2016

Publication series

NameProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Other

Other4th IEEE International Conference on Big Data, Big Data 2016
CountryUnited States
CityWashington
Period12/5/1612/8/16

Fingerprint

Supercomputers
Resource allocation
Data structures

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Hardware and Architecture

Cite this

Ueno, K., Suzumura, T., Maruyama, N., Fujisawa, K., & Matsuoka, S. (2016). Extreme scale breadth-first search on supercomputers. In R. Ak, G. Karypis, Y. Xia, X. T. Hu, P. S. Yu, J. Joshi, L. Ungar, L. Liu, A-H. Sato, T. Suzumura, S. Rachuri, R. Govindaraju, ... W. Xu (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 1040-1047). [7840705] (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840705

Extreme scale breadth-first search on supercomputers. / Ueno, Koji; Suzumura, Toyotaro; Maruyama, Naoya; Fujisawa, Katsuki; Matsuoka, Satoshi.

Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / Ronay Ak; George Karypis; Yinglong Xia; Xiaohua Tony Hu; Philip S. Yu; James Joshi; Lyle Ungar; Ling Liu; Aki-Hiro Sato; Toyotaro Suzumura; Sudarsan Rachuri; Rama Govindaraju; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1040-1047 7840705 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ueno, K, Suzumura, T, Maruyama, N, Fujisawa, K & Matsuoka, S 2016, Extreme scale breadth-first search on supercomputers. in R Ak, G Karypis, Y Xia, XT Hu, PS Yu, J Joshi, L Ungar, L Liu, A-H Sato, T Suzumura, S Rachuri, R Govindaraju & W Xu (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840705, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, Institute of Electrical and Electronics Engineers Inc., pp. 1040-1047, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 12/5/16. https://doi.org/10.1109/BigData.2016.7840705
Ueno K, Suzumura T, Maruyama N, Fujisawa K, Matsuoka S. Extreme scale breadth-first search on supercomputers. In Ak R, Karypis G, Xia Y, Hu XT, Yu PS, Joshi J, Ungar L, Liu L, Sato A-H, Suzumura T, Rachuri S, Govindaraju R, Xu W, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1040-1047. 7840705. (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). https://doi.org/10.1109/BigData.2016.7840705
Ueno, Koji ; Suzumura, Toyotaro ; Maruyama, Naoya ; Fujisawa, Katsuki ; Matsuoka, Satoshi. / Extreme scale breadth-first search on supercomputers. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / Ronay Ak ; George Karypis ; Yinglong Xia ; Xiaohua Tony Hu ; Philip S. Yu ; James Joshi ; Lyle Ungar ; Ling Liu ; Aki-Hiro Sato ; Toyotaro Suzumura ; Sudarsan Rachuri ; Rama Govindaraju ; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1040-1047 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).
@inproceedings{4ce7f0c76afa48988bb81c4ac41fa61a,
title = "Extreme scale breadth-first search on supercomputers",
abstract = "Breadth-First Search(BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid-BFS algorithm by Beamer[3], we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and optimization techniques such as vertex reordering and load balancing. Performance evaluation on the K shows our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously-known best techniques.",
author = "Koji Ueno and Toyotaro Suzumura and Naoya Maruyama and Katsuki Fujisawa and Satoshi Matsuoka",
year = "2016",
month = "1",
day = "1",
doi = "10.1109/BigData.2016.7840705",
language = "English",
series = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1040--1047",
editor = "Ronay Ak and George Karypis and Yinglong Xia and Hu, {Xiaohua Tony} and Yu, {Philip S.} and James Joshi and Lyle Ungar and Ling Liu and Aki-Hiro Sato and Toyotaro Suzumura and Sudarsan Rachuri and Rama Govindaraju and Weijia Xu",
booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",
address = "United States",

}

TY - GEN

T1 - Extreme scale breadth-first search on supercomputers

AU - Ueno, Koji

AU - Suzumura, Toyotaro

AU - Maruyama, Naoya

AU - Fujisawa, Katsuki

AU - Matsuoka, Satoshi

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Breadth-First Search(BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid-BFS algorithm by Beamer[3], we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and optimization techniques such as vertex reordering and load balancing. Performance evaluation on the K shows our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously-known best techniques.

AB - Breadth-First Search(BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid-BFS algorithm by Beamer[3], we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and optimization techniques such as vertex reordering and load balancing. Performance evaluation on the K shows our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously-known best techniques.

UR - http://www.scopus.com/inward/record.url?scp=85015250073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015250073&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840705

DO - 10.1109/BigData.2016.7840705

M3 - Conference contribution

T3 - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

SP - 1040

EP - 1047

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Ak, Ronay

A2 - Karypis, George

A2 - Xia, Yinglong

A2 - Hu, Xiaohua Tony

A2 - Yu, Philip S.

A2 - Joshi, James

A2 - Ungar, Lyle

A2 - Liu, Ling

A2 - Sato, Aki-Hiro

A2 - Suzumura, Toyotaro

A2 - Rachuri, Sudarsan

A2 - Govindaraju, Rama

A2 - Xu, Weijia

PB - Institute of Electrical and Electronics Engineers Inc.

ER -