Characterizing I/O and Storage Activity on the K Computer for Post-Processing Purposes

Eduardo C. Inacio, Jorji Nonaka, Kenji Ono, Mario A.R. Dantas, Fumiyoshi Shoji

    研究成果: Chapter in Book/Report/Conference proceedingConference contribution

    1 被引用数 (Scopus)

    抄録

    An increasing volume of data is produced by computational science applications executing on flagship-class supercomputers, such as the K computer. Most of these huge datasets would later pass through post-processing for visualization and analysis in order to derive meaningful information. Particular characteristics of the computing environment, application, and the dataset itself, can make efficiently exploring the performance capabilities of large-scale storage systems supporting these supercomputer a challenging task. This paper presents a characterization of the I/O and storage activity of jobs executed on the K computer focusing on post-processing purposes, based upon nine months of production operation recorded. Results demonstrate the intensive data demand of K computer applications, both in terms of volume of file I/O carried out during job execution, amount of data staged-in and staged-out, and number of files produced per job. These aspects shed light on challenges and opportunities for specialized data management libraries for posthoc data visualization and analysis.

    本文言語英語
    ホスト出版物のタイトル2018 IEEE Symposium on Computers and Communications, ISCC 2018
    出版社Institute of Electrical and Electronics Engineers Inc.
    ページ730-735
    ページ数6
    2018-June
    ISBN(電子版)9781538669501
    DOI
    出版ステータス出版済み - 11 15 2018
    イベント2018 IEEE Symposium on Computers and Communications, ISCC 2018 - Natal, ブラジル
    継続期間: 6 25 20186 28 2018

    出版物シリーズ

    名前Proceedings - IEEE Symposium on Computers and Communications
    2018-June
    ISSN(印刷版)1530-1346

    会議

    会議2018 IEEE Symposium on Computers and Communications, ISCC 2018
    国/地域ブラジル
    CityNatal
    Period6/25/186/28/18

    All Science Journal Classification (ASJC) codes

    • ソフトウェア
    • 信号処理
    • 数学 (全般)
    • コンピュータ サイエンスの応用
    • コンピュータ ネットワークおよび通信

    フィンガープリント

    「Characterizing I/O and Storage Activity on the K Computer for Post-Processing Purposes」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

    引用スタイル