Hybrid BFS approach using semi-external memory

Keita Iwabuchi, Hitoshi Sato, Ryo Mizote, Yuichiro Yasui, Katsuki Fujisawa, Satoshi Matsuoka

研究成果: 書籍/レポート タイプへの寄稿会議への寄与

6 被引用数 (Scopus)


NVM devices will greatly expand the possibility of processing extremely large-scale graphs that exceed the DRAM capacity of the nodes, however, efficient implementation based on detailed performance analysis of access patterns of unstructured graph kernel on systems that utilize a mixture of DRAM and NVM devices has not been well investigated. We introduce a graph data offloading technique using NVMs that augment the hybrid BFS (Breadth-first search) algorithm widely used in the Graph500 benchmark, and conduct performance analysis to demonstrate the utility of NVMs for unstructured data. Experimental results of a Scale27 problem of a Kronecker graph compliant to the Graph500 benchmark show that our approach maximally sustains 4.22 Giga TEPS (Traversed Edges Per Second), reducing DRAM size by half with only 19.18% performance degradation on a 4-way AMD Opteron 6172 machine heavily equipped with NVM devices. Although direct comparison is difficult, this is significantly greater than the result of 0.05 GTEPS for a SCALE 36 problem by using 1TB of DRAM and 12 TB of NVM as reported by Pearce et al. Although our approach uses higher DRAM to NVM ratio, we show that a good compromise is achievable between performance vs. capacity ratio for processing large-scale graphs. This result as well as detailed performance analysis of the proposed technique suggests that we can process extremely large-scale graphs per node with minimum performance degradation by carefully considering the data structures of a given graph and the access patterns to both DRAM and NVM devices. As a result, our implementation has achieved 4.35 MTEPS/W(Mega TEPS per Watt) and ranked 4th on November 2013 edition of the Green Graph500 list in the Big Data category by using only a single fat server heavily equipped with NVMs.

ホスト出版物のタイトルProceedings - IEEE 28th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2014
出版社IEEE Computer Society
出版ステータス出版済み - 11月 27 2014
イベント28th IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2014 - Phoenix, 米国
継続期間: 5月 19 20145月 23 2014


名前Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS


その他28th IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2014

!!!All Science Journal Classification (ASJC) codes

  • 計算理論と計算数学
  • コンピュータ ネットワークおよび通信
  • ハードウェアとアーキテクチャ
  • ソフトウェア


「Hybrid BFS approach using semi-external memory」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。