Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines

Koji Ueno, Toyotaro Suzumura, Naoya Maruyama, Katsuki Fujisawa, Satoshi Matsuoka

    Research output: Contribution to journalArticlepeer-review

    17 Citations (Scopus)

    Abstract

    There are many large-scale graphs in real world such as Web graphs and social graphs. The interest in large-scale graph analysis is growing in recent years. Breadth-First Search (BFS) is one of the most fundamental graph algorithms used as a component of many graph algorithms. Our new method for distributed parallel BFS can compute BFS for one trillion vertices graph within half a second, using large supercomputers such as the K-Computer. By the use of our proposed algorithm, the K-Computer was ranked 1st in Graph500 using all the 82,944 nodes available on June and November 2015 and June 2016 38,621.4 GTEPS. Based on the hybrid BFS algorithm by Beamer (Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW ’13, IEEE Computer Society, Washington, 2013), we devise sets of optimizations for scaling to extreme number of nodes, including a new efficient graph data structure and several optimization techniques such as vertex reordering and load balancing. Our performance evaluation on K-Computer shows that our new BFS is 3.19 times faster on 30,720 nodes than the base version using the previously known best techniques.

    Original languageEnglish
    Pages (from-to)22-35
    Number of pages14
    JournalData Science and Engineering
    Volume2
    Issue number1
    DOIs
    Publication statusPublished - Mar 1 2017

    All Science Journal Classification (ASJC) codes

    • Computational Mechanics
    • Computer Science Applications

    Fingerprint

    Dive into the research topics of 'Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines'. Together they form a unique fingerprint.

    Cite this