Netal: High-performance implementation of network analysis library considering computer memory hierarchy

Yuichiro Yasui, Katsuki Fujisawa, Kazushige Goto, Naoyuki Kamiyama, Mizuyo Talcamatsu

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The use of network analysis has increased in various fields. In particular, a lot of attention has been paid to centrality metrics using shortest paths, which require a comparatively smaller amount of computation, and the global characteristic features within the network. While theoretical and experimental progress has enabled greater control over networks, large amounts of computation required for dealing with large-scale networks is a major hurdle. This research is on high-performance network analysis considering the memory hierarchy in a computer; it targets extremely important kernel types called shortest paths and centrality. Our implementation, called NETAL (NETwork Analysis Library), can achieve high efficiency in parallel processing using many-core processors such as the AMD Opteron 6174, which has the NUMA architecture. We demonstrated through tests on real-world networks that NETAL is faster than previous implementations. In the all-pairs shortest paths for the weighted graph USA-road-d. NY. gr (n. -264K, m -734K), our implementatioll solved the shortest path distance labels in 44.4 seconds and the shortest paths with multiple predecessors in 411.2 seconds. Compared with the 9th DJMACS benchmark solver, our implementation is, respectively, 302.7 times and 32.7 times faster. NETAL succeeded in solving the shortest path distance labels for the USA-road-d.IJSA.gr (ii =24M, m =581V1) without preprocessing in 7.75 days. Numerical results showed that our implementation performance was 432.4 times that of the Δ-stepping algorithm and 228.9 times that of the 9th DIMACS benchmark solver. Furthermore, while GraphCT took 18 hours to compute the betweenness of web-BerkStan, our implementation computed multiple centrality metncs (closeness, graph, stress, and betweenness) simultaneously within 1 hour. A performance increase of 2.4-3.7 times compared with R-MAT graph was confirmed for SSCA#2.

Original languageEnglish
Pages (from-to)259-280
Number of pages22
JournalJournal of the Operations Research Society of Japan
Volume54
Issue number4
DOIs
Publication statusPublished - Dec 2011

Fingerprint

Shortest path
Network analysis
High performance
Graph
Centrality
Betweenness
Roads
Benchmark
Kernel
Parallel processing
World Wide Web
Closeness

All Science Journal Classification (ASJC) codes

  • Decision Sciences(all)
  • Management Science and Operations Research

Cite this

Netal : High-performance implementation of network analysis library considering computer memory hierarchy. / Yasui, Yuichiro; Fujisawa, Katsuki; Goto, Kazushige; Kamiyama, Naoyuki; Talcamatsu, Mizuyo.

In: Journal of the Operations Research Society of Japan, Vol. 54, No. 4, 12.2011, p. 259-280.

Research output: Contribution to journalArticle

@article{d2f22b97e0da4a8d904fdac450cd910a,
title = "Netal: High-performance implementation of network analysis library considering computer memory hierarchy",
abstract = "The use of network analysis has increased in various fields. In particular, a lot of attention has been paid to centrality metrics using shortest paths, which require a comparatively smaller amount of computation, and the global characteristic features within the network. While theoretical and experimental progress has enabled greater control over networks, large amounts of computation required for dealing with large-scale networks is a major hurdle. This research is on high-performance network analysis considering the memory hierarchy in a computer; it targets extremely important kernel types called shortest paths and centrality. Our implementation, called NETAL (NETwork Analysis Library), can achieve high efficiency in parallel processing using many-core processors such as the AMD Opteron 6174, which has the NUMA architecture. We demonstrated through tests on real-world networks that NETAL is faster than previous implementations. In the all-pairs shortest paths for the weighted graph USA-road-d. NY. gr (n. -264K, m -734K), our implementatioll solved the shortest path distance labels in 44.4 seconds and the shortest paths with multiple predecessors in 411.2 seconds. Compared with the 9th DJMACS benchmark solver, our implementation is, respectively, 302.7 times and 32.7 times faster. NETAL succeeded in solving the shortest path distance labels for the USA-road-d.IJSA.gr (ii =24M, m =581V1) without preprocessing in 7.75 days. Numerical results showed that our implementation performance was 432.4 times that of the Δ-stepping algorithm and 228.9 times that of the 9th DIMACS benchmark solver. Furthermore, while GraphCT took 18 hours to compute the betweenness of web-BerkStan, our implementation computed multiple centrality metncs (closeness, graph, stress, and betweenness) simultaneously within 1 hour. A performance increase of 2.4-3.7 times compared with R-MAT graph was confirmed for SSCA#2.",
author = "Yuichiro Yasui and Katsuki Fujisawa and Kazushige Goto and Naoyuki Kamiyama and Mizuyo Talcamatsu",
year = "2011",
month = "12",
doi = "10.15807/jorsj.54.259",
language = "English",
volume = "54",
pages = "259--280",
journal = "Journal of the Operations Research Society of Japan",
issn = "0453-4514",
publisher = "Operations Research Society of Japan",
number = "4",

}

TY - JOUR

T1 - Netal

T2 - High-performance implementation of network analysis library considering computer memory hierarchy

AU - Yasui, Yuichiro

AU - Fujisawa, Katsuki

AU - Goto, Kazushige

AU - Kamiyama, Naoyuki

AU - Talcamatsu, Mizuyo

PY - 2011/12

Y1 - 2011/12

N2 - The use of network analysis has increased in various fields. In particular, a lot of attention has been paid to centrality metrics using shortest paths, which require a comparatively smaller amount of computation, and the global characteristic features within the network. While theoretical and experimental progress has enabled greater control over networks, large amounts of computation required for dealing with large-scale networks is a major hurdle. This research is on high-performance network analysis considering the memory hierarchy in a computer; it targets extremely important kernel types called shortest paths and centrality. Our implementation, called NETAL (NETwork Analysis Library), can achieve high efficiency in parallel processing using many-core processors such as the AMD Opteron 6174, which has the NUMA architecture. We demonstrated through tests on real-world networks that NETAL is faster than previous implementations. In the all-pairs shortest paths for the weighted graph USA-road-d. NY. gr (n. -264K, m -734K), our implementatioll solved the shortest path distance labels in 44.4 seconds and the shortest paths with multiple predecessors in 411.2 seconds. Compared with the 9th DJMACS benchmark solver, our implementation is, respectively, 302.7 times and 32.7 times faster. NETAL succeeded in solving the shortest path distance labels for the USA-road-d.IJSA.gr (ii =24M, m =581V1) without preprocessing in 7.75 days. Numerical results showed that our implementation performance was 432.4 times that of the Δ-stepping algorithm and 228.9 times that of the 9th DIMACS benchmark solver. Furthermore, while GraphCT took 18 hours to compute the betweenness of web-BerkStan, our implementation computed multiple centrality metncs (closeness, graph, stress, and betweenness) simultaneously within 1 hour. A performance increase of 2.4-3.7 times compared with R-MAT graph was confirmed for SSCA#2.

AB - The use of network analysis has increased in various fields. In particular, a lot of attention has been paid to centrality metrics using shortest paths, which require a comparatively smaller amount of computation, and the global characteristic features within the network. While theoretical and experimental progress has enabled greater control over networks, large amounts of computation required for dealing with large-scale networks is a major hurdle. This research is on high-performance network analysis considering the memory hierarchy in a computer; it targets extremely important kernel types called shortest paths and centrality. Our implementation, called NETAL (NETwork Analysis Library), can achieve high efficiency in parallel processing using many-core processors such as the AMD Opteron 6174, which has the NUMA architecture. We demonstrated through tests on real-world networks that NETAL is faster than previous implementations. In the all-pairs shortest paths for the weighted graph USA-road-d. NY. gr (n. -264K, m -734K), our implementatioll solved the shortest path distance labels in 44.4 seconds and the shortest paths with multiple predecessors in 411.2 seconds. Compared with the 9th DJMACS benchmark solver, our implementation is, respectively, 302.7 times and 32.7 times faster. NETAL succeeded in solving the shortest path distance labels for the USA-road-d.IJSA.gr (ii =24M, m =581V1) without preprocessing in 7.75 days. Numerical results showed that our implementation performance was 432.4 times that of the Δ-stepping algorithm and 228.9 times that of the 9th DIMACS benchmark solver. Furthermore, while GraphCT took 18 hours to compute the betweenness of web-BerkStan, our implementation computed multiple centrality metncs (closeness, graph, stress, and betweenness) simultaneously within 1 hour. A performance increase of 2.4-3.7 times compared with R-MAT graph was confirmed for SSCA#2.

UR - http://www.scopus.com/inward/record.url?scp=84880400762&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880400762&partnerID=8YFLogxK

U2 - 10.15807/jorsj.54.259

DO - 10.15807/jorsj.54.259

M3 - Article

AN - SCOPUS:84880400762

VL - 54

SP - 259

EP - 280

JO - Journal of the Operations Research Society of Japan

JF - Journal of the Operations Research Society of Japan

SN - 0453-4514

IS - 4

ER -