Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer

Chongke Bi, Kenji Ono, Kwan Liu Ma, Haiyuan Wu, Toshiyuki Imamura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

Original languageEnglish
Title of host publicationIEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings
PublisherIEEE Computer Society
Pages121-122
Number of pages2
DOIs
Publication statusPublished - Jan 1 2013
Externally publishedYes
Event2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013 - Atlanta, GA, United States
Duration: Oct 13 2013Oct 14 2013

Other

Other2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013
CountryUnited States
CityAtlanta, GA
Period10/13/1310/14/13

Fingerprint

Visualization
Supercomputers
Decomposition
Volume rendering
Binary images
Costs
Data storage equipment
Communication
Processing
Chemical analysis
Big data

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition

Cite this

Bi, C., Ono, K., Ma, K. L., Wu, H., & Imamura, T. (2013). Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. In IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings (pp. 121-122). [6675169] IEEE Computer Society. https://doi.org/10.1109/LDAV.2013.6675169

Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. / Bi, Chongke; Ono, Kenji; Ma, Kwan Liu; Wu, Haiyuan; Imamura, Toshiyuki.

IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society, 2013. p. 121-122 6675169.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bi, C, Ono, K, Ma, KL, Wu, H & Imamura, T 2013, Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. in IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings., 6675169, IEEE Computer Society, pp. 121-122, 2013 3rd IEEE Symposium on Large-Scale Data Analysis and Visualization, LDAV 2013, Atlanta, GA, United States, 10/13/13. https://doi.org/10.1109/LDAV.2013.6675169
Bi C, Ono K, Ma KL, Wu H, Imamura T. Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. In IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society. 2013. p. 121-122. 6675169 https://doi.org/10.1109/LDAV.2013.6675169
Bi, Chongke ; Ono, Kenji ; Ma, Kwan Liu ; Wu, Haiyuan ; Imamura, Toshiyuki. / Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer. IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings. IEEE Computer Society, 2013. pp. 121-122
@inproceedings{3d563f34c2c945fa90eb97f90474baba,
title = "Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer",
abstract = "The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.",
author = "Chongke Bi and Kenji Ono and Ma, {Kwan Liu} and Haiyuan Wu and Toshiyuki Imamura",
year = "2013",
month = "1",
day = "1",
doi = "10.1109/LDAV.2013.6675169",
language = "English",
pages = "121--122",
booktitle = "IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings",
publisher = "IEEE Computer Society",
address = "United States",

}

TY - GEN

T1 - Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer

AU - Bi, Chongke

AU - Ono, Kenji

AU - Ma, Kwan Liu

AU - Wu, Haiyuan

AU - Imamura, Toshiyuki

PY - 2013/1/1

Y1 - 2013/1/1

N2 - The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

AB - The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.

UR - http://www.scopus.com/inward/record.url?scp=84899128400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899128400&partnerID=8YFLogxK

U2 - 10.1109/LDAV.2013.6675169

DO - 10.1109/LDAV.2013.6675169

M3 - Conference contribution

AN - SCOPUS:84899128400

SP - 121

EP - 122

BT - IEEE Symposium on Large Data Analysis and Visualization 2013, LDAV 2013 - Proceedings

PB - IEEE Computer Society

ER -