Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer

Chongke Bi, Kenji Ono, Kwan Liu Ma, Haiyuan Wu, Toshiyuki Imamura

研究成果: 著書/レポートタイプへの貢献会議での発言

6 引用 (Scopus)

抄録

The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

元の言語英語
ホスト出版物のタイトルIEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings
出版者IEEE Computer Society
ページ121-122
ページ数2
DOI
出版物ステータス出版済み - 1 1 2013
外部発表Yes
イベント2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013 - Atlanta, GA, 米国
継続期間: 10 13 201310 14 2013

その他

その他2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013
米国
Atlanta, GA
期間10/13/1310/14/13

Fingerprint

Visualization
Supercomputers
Decomposition
Volume rendering
Binary images
Costs
Data storage equipment
Communication
Processing
Chemical analysis
Big data

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition

これを引用

Bi, C., Ono, K., Ma, K. L., Wu, H., & Imamura, T. (2013). Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. : IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings (pp. 121-122). [6675169] IEEE Computer Society. https://doi.org/10.1109/LDAV.2013.6675169

Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. / Bi, Chongke; Ono, Kenji; Ma, Kwan Liu; Wu, Haiyuan; Imamura, Toshiyuki.

IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society, 2013. p. 121-122 6675169.

研究成果: 著書/レポートタイプへの貢献会議での発言

Bi, C, Ono, K, Ma, KL, Wu, H & Imamura, T 2013, Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. : IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings., 6675169, IEEE Computer Society, pp. 121-122, 2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013, Atlanta, GA, 米国, 10/13/13. https://doi.org/10.1109/LDAV.2013.6675169
Bi C, Ono K, Ma KL, Wu H, Imamura T. Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. : IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society. 2013. p. 121-122. 6675169 https://doi.org/10.1109/LDAV.2013.6675169
Bi, Chongke ; Ono, Kenji ; Ma, Kwan Liu ; Wu, Haiyuan ; Imamura, Toshiyuki. / Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society, 2013. pp. 121-122
@inproceedings{3d563f34c2c945fa90eb97f90474baba,
title = "Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer",
abstract = "The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.",
author = "Chongke Bi and Kenji Ono and Ma, {Kwan Liu} and Haiyuan Wu and Toshiyuki Imamura",
year = "2013",
month = "1",
day = "1",
doi = "10.1109/LDAV.2013.6675169",
language = "English",
pages = "121--122",
booktitle = "IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings",
publisher = "IEEE Computer Society",
address = "United States",

}

TY - GEN

T1 - Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer

AU - Bi, Chongke

AU - Ono, Kenji

AU - Ma, Kwan Liu

AU - Wu, Haiyuan

AU - Imamura, Toshiyuki

PY - 2013/1/1

Y1 - 2013/1/1

N2 - The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

AB - The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

UR - http://www.scopus.com/inward/record.url?scp=84899128400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899128400&partnerID=8YFLogxK

U2 - 10.1109/LDAV.2013.6675169

DO - 10.1109/LDAV.2013.6675169

M3 - Conference contribution

AN - SCOPUS:84899128400

SP - 121

EP - 122

BT - IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings

PB - IEEE Computer Society

ER -