Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets

Kenji Ono, Jorji Nonaka, Hiroyuki Yoshikawa, Takeshi Nanri, Yoshiyuki Morie, Tomohiro Kawanabe, Fumiyoshi Shoji

研究成果: 著書/レポートタイプへの貢献会議での発言

抄録

This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.

元の言語英語
ホスト出版物のタイトルHigh Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers
編集者Michèle Weiland, Sadaf Alam, Rio Yokota, John Shalf
出版者Springer Verlag
ページ243-257
ページ数15
ISBN(印刷物)9783030024642
DOI
出版物ステータス出版済み - 1 1 2018
イベントInternational Conference on High Performance Computing, ISC High Performance 2018 - Frankfurt, ドイツ
継続期間: 6 28 20186 28 2018

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
11203 LNCS
ISSN(印刷物)0302-9743
ISSN(電子版)1611-3349

会議

会議International Conference on High Performance Computing, ISC High Performance 2018
ドイツ
Frankfurt
期間6/28/186/28/18

Fingerprint

Data visualization
Data Visualization
Buffer
Time-varying
Python
Simulation
Visualization
Simulators
Data storage equipment
Communication
Processing
Initialization
Post-processing
Proximity
Preprocessing
Simulator
Flexibility
Design
Framework
Prototype

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

これを引用

Ono, K., Nonaka, J., Yoshikawa, H., Nanri, T., Morie, Y., Kawanabe, T., & Shoji, F. (2018). Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. : M. Weiland, S. Alam, R. Yokota, & J. Shalf (版), High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers (pp. 243-257). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻数 11203 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-02465-9_17

Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. / Ono, Kenji; Nonaka, Jorji; Yoshikawa, Hiroyuki; Nanri, Takeshi; Morie, Yoshiyuki; Kawanabe, Tomohiro; Shoji, Fumiyoshi.

High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers. 版 / Michèle Weiland; Sadaf Alam; Rio Yokota; John Shalf. Springer Verlag, 2018. p. 243-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻 11203 LNCS).

研究成果: 著書/レポートタイプへの貢献会議での発言

Ono, K, Nonaka, J, Yoshikawa, H, Nanri, T, Morie, Y, Kawanabe, T & Shoji, F 2018, Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. : M Weiland, S Alam, R Yokota & J Shalf (版), High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 巻. 11203 LNCS, Springer Verlag, pp. 243-257, International Conference on High Performance Computing, ISC High Performance 2018, Frankfurt, ドイツ, 6/28/18. https://doi.org/10.1007/978-3-030-02465-9_17
Ono K, Nonaka J, Yoshikawa H, Nanri T, Morie Y, Kawanabe T その他. Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. : Weiland M, Alam S, Yokota R, Shalf J, 編集者, High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers. Springer Verlag. 2018. p. 243-257. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-02465-9_17
Ono, Kenji ; Nonaka, Jorji ; Yoshikawa, Hiroyuki ; Nanri, Takeshi ; Morie, Yoshiyuki ; Kawanabe, Tomohiro ; Shoji, Fumiyoshi. / Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets. High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers. 編集者 / Michèle Weiland ; Sadaf Alam ; Rio Yokota ; John Shalf. Springer Verlag, 2018. pp. 243-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{a5b19c86ace34952ac03fa6e7114a2d3,
title = "Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets",
abstract = "This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.",
author = "Kenji Ono and Jorji Nonaka and Hiroyuki Yoshikawa and Takeshi Nanri and Yoshiyuki Morie and Tomohiro Kawanabe and Fumiyoshi Shoji",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-02465-9_17",
language = "English",
isbn = "9783030024642",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "243--257",
editor = "Mich{\`e}le Weiland and Sadaf Alam and Rio Yokota and John Shalf",
booktitle = "High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers",
address = "Germany",

}

TY - GEN

T1 - Design of a Flexible In Situ Framework with a Temporal Buffer for Data Processing and Visualization of Time-Varying Datasets

AU - Ono, Kenji

AU - Nonaka, Jorji

AU - Yoshikawa, Hiroyuki

AU - Nanri, Takeshi

AU - Morie, Yoshiyuki

AU - Kawanabe, Tomohiro

AU - Shoji, Fumiyoshi

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.

AB - This paper presents an in situ framework focused on time-varying simulations, and uses a novel temporal buffer for storing simulation results sampled at user-defined intervals. This framework has been designed to provide flexible data processing and visualization capabilities in modern HPC operational environments composed of powerful front-end systems, for pre-and post-processing purposes, along with traditional back-end HPC systems. The temporal buffer is implemented using the functionalities provided by Open Address Space (OpAS) library, which enables asynchronous one-sided communication from outside processes to any exposed memory region on the simulator side. This buffer can store time-varying simulation results, and can be processed via in situ approaches with different proximities. We present a prototype of our framework, and code integration process with a target simulation code. The proposed in situ framework utilizes separate files to describe the initialization and execution codes, which are in the form of Python scripts. This framework also enables the runtime modification of these Python-based files, thus providing greater flexibility to the users, not only for data processing, such as visualization and analysis, but also for the simulation steering.

UR - http://www.scopus.com/inward/record.url?scp=85066148704&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066148704&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-02465-9_17

DO - 10.1007/978-3-030-02465-9_17

M3 - Conference contribution

AN - SCOPUS:85066148704

SN - 9783030024642

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 243

EP - 257

BT - High Performance Computing - ISC High Performance 2018 International Workshops, Revised Selected Papers

A2 - Weiland, Michèle

A2 - Alam, Sadaf

A2 - Yokota, Rio

A2 - Shalf, John

PB - Springer Verlag

ER -